cdp_backend.file_store package#
Submodules#
cdp_backend.file_store.functions module#
- cdp_backend.file_store.functions.download_file(credentials_file: str, bucket: str, remote_filepath: str, save_path: str) str[source]#
- cdp_backend.file_store.functions.get_file_uri(bucket: str, filename: str, credentials_file: str) str | None[source]#
Gets the file uri of a filename and bucket for a given Google Cloud file store.
- Parameters:
- bucket: str
The bucket name to check for the file.
- filename: str
The filename of the file to check for.
- credentials_file: str
The path to the Google Service Account credentials JSON file used to initialize the file store connection.
- Returns:
- file_uri: Optional[str]
The file uri if the file exists, otherwise returns None.
- cdp_backend.file_store.functions.get_open_url_for_gcs_file(credentials_file: str, uri: str) str[source]#
Simple wrapper around fsspec.FileSystem.url function for creating a connection to the filesystem then getting the hosted / web accessible URL to the file.
- Parameters:
- credentials_file: str
The path to the Google Service Account credentials JSON file used to initialize the file store connection.
- uri: str
The URI to the file already stored to get a web accessible URL for.
- Returns:
- url: str
The web accessible URL for the file.
- cdp_backend.file_store.functions.initialize_gcs_file_system(credentials_file: str) GCSFileSystem[source]#
Initializes an instance of a GCSFileSystem.
- Parameters:
- credentials_file: str
The path to the Google Service Account credentials JSON file.
- Returns:
- file_system: GCSFileSystem
An initialized GCSFileSystem.
- cdp_backend.file_store.functions.remove_local_file(filepath: str | Path) None[source]#
Deletes a file from the local file system.
- Parameters:
- filepath: str
The filepath of the local file to delete.
- cdp_backend.file_store.functions.upload_file(credentials_file: str, bucket: str, filepath: str, save_name: str | None = None, remove_local: bool = False, overwrite: bool = False) str[source]#
Uploads a file to a Google Cloud file store bucket. If save_name is provided, that will be used as the file’s save name instead of the file’s local name. If remove_local is provided, the local file will be removed upon successful upload.
- Parameters:
- credentials_file: str
The path to the Google Service Account credentials JSON file used to initialize the file store connection.
- bucket: str
The name of the file store bucket to upload to.
- filepath: str
The filepath to the local file to upload.
- save_name: Optional[str]
The name to save the file as in the file store.
- remove_local: bool
If True, remove the local file upon successful upload.
- overwrite: bool
Boolean value indicating whether or not to overwrite the remote resource with the same name if it already exists.
- Returns:
- uri: str
The uri of the uploaded file in the file store.
- cdp_backend.file_store.functions.upload_file_and_return_link(credentials_file: str, bucket: str, filepath: str, save_name: str | None = None, remove_local: bool = False) str[source]#
- Parameters:
- credentials_file: str
The path to the Google Service Account credentials JSON file used to initialize the file store connection.
- bucket: str
The name of the file store bucket to upload to.
- filepath: str
The filepath to the local file to upload.
- save_name: Optional[str]
The name to save the file as in the file store.
- remove_local: bool
If True, remove the local file upon successful upload.
- Returns:
- str
HTTPS link to the uploaded file.
Module contents#
File store package for cdp_backend.