plate_model_manager package

Submodules

plate_model_manager.download_utils module

plate_model_manager.download_utils.check_redownload_need(metadata_file, url)

check the metadata file and decide if redownload is necessary

Parameters:

• metadata_file – metadata file path

• url – url for the target file

Returns download_flag, etag:

a flag indicates if redownload is neccesarry and old etag in the meta file.

plate_model_manager.download_utils.download_file(url, metadata_file, dst_path, expire_hours=12, large_file_hint=False)

download a file from “url”, save the file in “dst_path” and write the metadata a metadata file will also be created for the file

Parameters:

• url – the url to the raster file

• metadata_file – the path to the metadata

• dst_path – the folder path to save the raster file

plate_model_manager.file_fetcher module

class plate_model_manager.file_fetcher.FileFetcher

Bases: object

abstract fetch_file(url: str, filepath: str, filename: str = None, etag: str = None, auto_unzip: bool = True)

download a file from “url” and save to “filepath” You can give a new “filename” for the file. If “etag” is given, check if etag has changed. If not changed, do not download again.

Parameters:

• url – the url to download file from

• filepath – location to keep the file

• filename – new file name (optional)

• etag – old etag. if the old etag is the same with the one on server, do not download again.

• auto_unzip – bool flag to indicate if unzip .zip file automatically

Returns:

new etag

abstract fetch_files(filepaths, filenames=[], etags=[], auto_unzip: bool = True)

fetch multiple files concurrently

Parameters:

• urls – the urls to download files from

• filepaths – location(s) to keep the files. This can be one path for all files or one path for each file.

• filenames – new file names (optional)

• etags – old etags. if the old etag is the same with the one on server, do not download again.

• auto_unzip – bool flag to indicate if unzip .zip file automatically

fetch_large_file(url: str, filepath: str, filesize: int = None, etag: str = None, auto_unzip: bool = True, check_etag: bool = True)

use multi-thread to fetch a large file.

LOOK HERE!!! Be careful when use this function. You cannot get partial content if the content is gzip encoded. So the file might be larger than the one download directly. It is useful when downloading large .zip file. Warning: this could be slower than single thread download. Some firewall sequences these requests to shape network traffic and defeat the purpose of this function completely.

check the etag and get content-length before calling this function

Parameters:

• url – the file url

• filepath – location to keep the file

• filesize – the size of file (in bytes)

• etag – old etag. if the old etag is the same with the one on server, do not download again.

• auto_unzip – bool flag to indicate if unzip .zip file automatically

Returns:

new etag

plate_model_manager.misc_utils module

plate_model_manager.misc_utils.my_warningformat(message, category, filename, lineno, line=None)

plate_model_manager.misc_utils.print_error(msg)

plate_model_manager.misc_utils.print_warning(msg)

plate_model_manager.network_aiohttp module

class plate_model_manager.network_aiohttp.AiohttpFetcher

Bases: FileFetcher

fetch_file(url: str, filepath: str, etag: str = None, auto_unzip: bool = True)

download a file from “url” and save to “filepath”

You can give a new “filename” for the file. If “etag” is given, check if etag has changed. If not changed, do not download again.

Parameters:

• url – the url to download file from

• filepath – location to keep the file

• etag – old etag. if the old etag is the same with the one on server, do not download again.

• auto_unzip – bool flag to indicate if unzip .zip file automatically

fetch_files(urls, filepaths, etags=[], auto_unzip: bool = True)

fetch multiple files concurrently

Parameters:

• urls – the urls to download files from

• filepaths – location(s) to keep the files. This can be one path for all files or one path for each file.

• etags – old etags. if the old etag is the same with the one on server, do not download again.

• auto_unzip – bool flag to indicate if unzip .zip file automatically

plate_model_manager.network_aiohttp.fetch_file(url: str, filepath: str, etag: str = None, auto_unzip: bool = True)

plate_model_manager.network_aiohttp.fetch_files(urls, filepaths, etags=[], auto_unzip: bool = True)

plate_model_manager.network_aiohttp.fetch_large_file(url: str, filepath: str, filesize: int = None, etag: str = None, auto_unzip: bool = True, check_etag: bool = True)

plate_model_manager.network_requests module

class plate_model_manager.network_requests.RequestsFetcher

Bases: FileFetcher

fetch_file(url: str, filepath: str, filename: str = None, etag: str = None, auto_unzip: bool = True)

download a file from “url” and save to “filepath”

You can give a new “filename” for the file. If “etag” is given, check if etag has changed. If not changed, do not download again.

Parameters:

• url – the url to download file from

• filepath – location to keep the file

• etag – old etag. if the old etag is the same with the one on server, do not download again.

• auto_unzip – bool flag to indicate if unzip .zip file automatically

fetch_files(urls, filepaths: list | str, etags=[], auto_unzip: bool = True)

fetch multiple files concurrently

Parameters:

• urls – the urls to download files from

• filepaths – location(s) to keep the files. This can be one path for all files or one path for each file.

• etags – old etags. if the old etag is the same with the one on server, do not download again.

• auto_unzip – bool flag to indicate if unzip .zip file automatically

plate_model_manager.network_requests.fetch_file(url: str, filepath: str, filename: str = None, etag: str = None, auto_unzip: bool = True)

plate_model_manager.network_requests.fetch_files(urls, filepaths: list | str, etags=[], auto_unzip: bool = True)

plate_model_manager.network_requests.fetch_large_file(url: str, filepath: str, filesize: int = None, etag: str = None, auto_unzip: bool = True, check_etag: bool = True)

plate_model_manager.network_utils module

plate_model_manager.network_utils.get_content_length(headers)

plate_model_manager.network_utils.get_etag(headers)

return the etag in the headers. The return could be none if the server does not support etag.

Parameters:

headers – call get_headers(url) to get headers

plate_model_manager.network_utils.get_headers(url)

plate_model_manager.plate_model module

class plate_model_manager.plate_model.PlateModel(model_name: str, model_cfg=None, data_dir='.', readonly=False)

Bases: object

Class to manage a plate model

create_model_dir()

create a model folder with a .metadata.json file in it

download_all()

download everything in this plate model

download_all_layers()

download all layers. Call download_layer_files() on every layer

download_layer_files(layer_name)

given the layer name, download the layer files. The layer files are in a .zip file. download and unzip it.

Parameters:

layer_name – such as “Rotations”,”Coastlines”, “StaticPolygons”, “ContinentalPolygons”, “Topologies”, etc

Returns:

the folder path which contains the layer files

download_raster(url, dst_path)

download a single raster file from “url” and save the file in “dst_path” a metadata file will also be created for the raster file in folder f”{dst_path}/metadata”

Parameters:

• url – the url to the raster file

• dst_path – the folder path to save the raster file

download_time_dependent_rasters(raster_name, times=None)

download time dependent rasters, such agegrids

Parameters:

• raster_name – raster name, such as AgeGrids. see the models.json

• times – if not given, download from begin to end with 1My interval

get_COBs()

return COBs feature collection

get_avail_layers()

get all available layers in this plate model

get_avail_time_dependent_raster_names()

return the names of all time dependent rasters which have been configurated in this model.

get_big_time()

get_cfg()

get_coastlines()

return coastlines feature collection

get_continental_polygons()

return ContinentalPolygons feature collection

get_data_dir()

get_layer(layer_name)

get layer files by name

Parameters:

layer_name – layer name

Returns:

a list of file names

get_model_dir()

get_raster(raster_name, time)

return a local path for the raster

Returns:

a local path of the raster file

get_rasters(raster_name, times)

return local paths for the raster files

Parameters:

times – a list of times

Returns:

a list of local paths

get_rotation_model()

return a list of rotation files

get_small_time()

get_static_polygons()

return StaticPolygons feature collection

get_topologies()

return Topologies feature collection

static is_model_dir(folder_path)

return True if it is a model dir, otherwise False

purge()

remove the model folder and everything inside it

purge_layer(layer_name)

remove the layer folder of the given layer name

purge_time_dependent_rasters(raster_name)

remove the raster folder of the given raster name

set_data_dir(new_dir)

plate_model_manager.plate_model_manager module

class plate_model_manager.plate_model_manager.PlateModelManager(model_manifest=None)

Bases: object

load a models.json file and manage plate models see an example models.json file at PlateModelManager.get_default_repo_url()

download_all_models(data_dir='./')

download all available models into data_dir

get_available_model_names()

return the names of available models as a list

static get_default_repo_url()

static get_local_available_model_names(local_dir)

list all model names in a local folder

get_model(model_name: str = 'default', data_dir: str = '.')

return a PlateModel object by model_name

Parameters:

• model_name – model name

• data_dir – the default data_dir for the model. This dir can be changed with PlateModel.set_data_dir() later.

Returns:

a PlateModel object or none if model name is no good

plate_model_manager.present_day_rasters module

class plate_model_manager.present_day_rasters.PresentDayRasterManager(raster_manifest=None)

Bases: object

manage present-day rasters

get_raster(_name: str, width=1800, height=800, bbox=[-180, -80, 180, 80])

download the raster by name. Save the raster in self.data_dir

is_wms(_name: str, check_raster_avail_flag=True)

return True if the raster is served by Web Map Service, otherwise False

list_present_day_rasters()

set_data_dir(folder)

exception plate_model_manager.present_day_rasters.RasterNameNotFound

Bases: Exception

plate_model_manager.unzip_utils module

plate_model_manager.unzip_utils.save_compressed_data(url, data, dst_path)

extract files from compressed data

Parameters:

• url – URL

• data – bytes-like object

• dst_path – location to save the files

plate_model_manager.zenodo module

class plate_model_manager.zenodo.ZenodoRecord(conceptrecid)

Bases: object

get_all_version_ids()

get_all_versions()

get_file_links(id)

get_filenames(id)

get_latest_version()

get_latest_version_id()

get_version(id)

Module contents