ephemeris package

Submodules

ephemeris.common_parser module

ephemeris.common_parser.get_common_args(login_required=True, log_file=False)

ephemeris.ephemeris_log module

class ephemeris.ephemeris_log.ProgressConsoleHandler(stream=None)

Bases: logging.StreamHandler

A handler class which allows the cursor to stay on one line for selected messages

emit(record)

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

on_same_line = False

ephemeris.ephemeris_log.disable_external_library_logging()

ephemeris.ephemeris_log.setup_global_logger(name, log_file=None)

ephemeris.generate_tool_list_from_ga_workflow_files module

Tool to generate tools from workflows

ephemeris.generate_tool_list_from_ga_workflow_files.generate_tool_list_from_workflow(workflow_files, panel_label, output_file)

Return type:object

ephemeris.generate_tool_list_from_ga_workflow_files.get_workflow_dictionary(json_file)

ephemeris.generate_tool_list_from_ga_workflow_files.main()

ephemeris.generate_tool_list_from_ga_workflow_files.print_yaml_tool_list(tool_dictionary, output_file)

ephemeris.generate_tool_list_from_ga_workflow_files.reduce_tool_list(tool_list)

ephemeris.generate_tool_list_from_ga_workflow_files.translate_workflow_dictionary_to_tool_list(tool_dictionary, panel_label)

ephemeris.get_tool_list_from_galaxy module

Tool to extract a tool list from galaxy.

class ephemeris.get_tool_list_from_galaxy.GiToToolYaml(gi, include_tool_panel_section_id=False, skip_tool_panel_section_name=True, skip_changeset_revision=False, get_data_managers=False, get_all_tools=False)

filter_section_name_or_id_or_changeset(repository_list)

installed_tool_list

gets a tool list from the toolclient :return:

repository_list

Toolbox elements returned by api/tools may be of class ToolSection or Tool. Parse these accordingly to get a list of repositories.

tool_list

toolbox

Gets the toolbox elements from <galaxy_url>/api/tools

write_to_yaml(output_file)

ephemeris.get_tool_list_from_galaxy.check_galaxy_version(gi)

ephemeris.get_tool_list_from_galaxy.get_repo_from_tool(tool)

Get the minimum items required for re-installing a (list of) tools

ephemeris.get_tool_list_from_galaxy.get_tool_panel(gi)

ephemeris.get_tool_list_from_galaxy.main()

ephemeris.get_tool_list_from_galaxy.merge_repository_changeset_revisions(repository_list)

Each installed changeset revision of a tool is listed individually. Merge revisions of the same tool into a list.

ephemeris.get_tool_list_from_galaxy.the_same_repository(repo_1_info, repo_2_info, check_revision=True)

Given two dicts containing info about repositories, determine if they are the same repository. Each of the dicts must have the following keys: changeset_revisions`( if check revisions is true), `name, owner, and (either tool_shed or tool_shed_url).

ephemeris.get_tool_list_from_galaxy.tools_for_repository(gi, repository)

ephemeris.get_tool_list_from_galaxy.walk_tools(tool_panel, f)

ephemeris.run_data_managers module

Run-data-managers is a tool for provisioning data on a galaxy instance.

Run-data-managers has the ability to run multiple data managers that are interdependent. When a reference genome is needed for bwa-mem for example, Run-data-managers can first run a data manager to fetch the fasta file and run another data manager that indexes the fasta file for bwa-mem. This functionality depends on the “watch_tool_data_dir” setting in galaxy.ini to be True. Also, if a new data manager is installed, galaxy needs to be restarted in order for it’s tool_data_dir to be watched.

Run-data-managers needs a yaml that specifies what data managers are run and with which settings. Example files can be found here, here, and here.

By default run-data-managers skips entries in the yaml file that have already been run. It checks it in the following way: * If the data manager has input variables “name” or “sequence_name” it will check if the “name” column in the data table already has this entry.

“name” will take precedence over “sequence_name”.

• If the data manager has input variables “value”, “sequence_id” or ‘dbkey’ it will check if the “value” column in the data table already has this entry. Value takes precedence over sequence_id which takes precedence over dbkey.
• If none of the above input variables are specified the data manager will always run.

class ephemeris.run_data_managers.DataManagers(galaxy_instance, configuration)

data_table_entry_exists(data_table_name, entry, column='value')

Checks whether an entry exists in the a specified column in the data_table.

dm_is_fetcher(dm)

Checks whether the data manager fetches a sequence instead of indexing. This is based on the source table. :returns True if dm is a fetcher. False if it is not.

get_dm_jobs(dm)

Gets the job entries for a single dm. Puts entries that already present in skipped_job_list. :returns job_list, skipped_job_list

initiate_job_lists()

Determines which data managers should be run to populate the data tables. Distinguishes between fetch jobs (download files) and index jobs. :return: populate self.fetch_jobs, self.skipped_fetch_jobs, self.index_jobs and self.skipped_index_jobs

input_entries_exist_in_data_tables(data_tables, input_dict)

Checks whether name and value entries from the input are already present in the data tables. If an entry is missing in of the tables, this function returns False

parse_items(items)

Parses items with jinja2. :param items: the items to be parsed :return: the parsed items

run(log, ignore_errors=False, overwrite=False)

Runs the data managers. :param log: The log to be used. :param ignore_errors: Ignore erroring data_managers. Continue regardless. :param overwrite: Overwrite existing entries in data tables

ephemeris.run_data_managers.get_first_valid_entry(input_dict, key_list)

Iterates over key_list and returns the value of the first key that exists in the dictionary. Or returns None

ephemeris.run_data_managers.main()

ephemeris.run_data_managers.wait(gi, job_list, log)

Waits until all jobs in a list are finished or failed. It will check the state of the created datasets every 30s. It will return a tuple: ( finished_jobs, failed_jobs )

ephemeris.setup_data_libraries module

Tool to setup data libraries on a galaxy instance

ephemeris.setup_data_libraries.create_batch_api(gi, desc)

ephemeris.setup_data_libraries.create_legacy(gi, desc)

ephemeris.setup_data_libraries.main()

ephemeris.setup_data_libraries.setup_data_libraries(gi, data, training=False, legacy=False)

Load files into a Galaxy data library. By default all test-data tools from all installed tools will be linked into a data library.

ephemeris.shed_tools module

A tool to automate installation of tool repositories from a Galaxy Tool Shed into an instance of Galaxy.

Shed-tools has three commands: update, test and install.

Update simply updates all the tools in a Galaxy given connection details on the command line.

Test tests the specified tools in the Galaxy Instance.

Install allows installation of tools in multiple ways. Galaxy instance details and the installed tools can be provided in one of three ways:

1. In the YAML format via dedicated files (a sample can be found here).
2. On the command line as dedicated script options (see the usage help).
3. As a single composite parameter to the script. The parameter must be a single, YAML-formatted string with the keys corresponding to the keys available for use in the YAML formatted file (for example: –yaml_tool “{‘owner’: ‘kellrott’, ‘tool_shed_url’: ‘https://testtoolshed.g2.bx.psu.edu’, ‘tool_panel_section_id’: ‘peak_calling’, ‘name’: ‘synapse_interface’}”).

Only one of the methods can be used with each invocation of the script but if more than one are provided are provided, precedence will correspond to order of the items in the list above. When installing tools, Galaxy expects any tool_panel_section_id provided when installing a tool to already exist in the configuration. If the section does not exist, the tool will be installed outside any section. See shed_tool_conf.xml.sample in this directory for a sample of such file. Before running this script to install the tools, make sure to place such file into Galaxy’s configuration directory and set Galaxy configuration option tool_config_file to include it.

class ephemeris.shed_tools.InstallRepositoryManager(galaxy_instance)

Bases: object

Manages the installation of new repositories on a galaxy instance

filter_installed_repos(repos, check_revision=True)

This filters a list of repositories

install_repositories(repositories, log=None, force_latest_revision=False, default_toolshed='https://toolshed.g2.bx.psu.edu/', default_install_tool_dependencies=False, default_install_resolver_dependencies=True, default_install_repository_dependencies=True)

Install a list of tools on the current galaxy

install_repository_revision(repository, log)

installed_repositories()

Get currently installed tools

test_tools(test_json, repositories=None, log=None, test_user_api_key=None, test_user='ephemeris@galaxyproject.org')

Run tool tests for all tools in each repository in supplied tool list or self.installed_repositories().

update_repositories(repositories=None, log=None, **kwargs)

wait_for_install(repository, log=None, timeout=3600)

If nginx times out, we look into the list of installed repositories and try to determine if a repository of the same namer/owner is still installing. Returns True if install finished successfully, returns False when timeout is exceeded or installation has failed.

ephemeris.shed_tools.args_to_repos(args)

ephemeris.shed_tools.log_repository_install_error(repository, start, msg, log)

Log failed repository installations. Return a dictionary with information

ephemeris.shed_tools.log_repository_install_skip(repository, counter, total_num_repositories, log)

ephemeris.shed_tools.log_repository_install_start(repository, counter, total_num_repositories, installation_start, log)

ephemeris.shed_tools.log_repository_install_success(repository, start, log)

Log successful repository installation. Repositories that finish in error still count as successful installs currently.

ephemeris.shed_tools.main()

ephemeris.sleep module

Utility to do a blocking sleep until a Galaxy instance is responsive. This is useful in docker images, in RUN steps, where one needs to wait for a currently starting Galaxy to be alive, before API requests can be made successfully.

The script functions by making repeated requests to http(s)://fqdn/api/version, an API which requires no authentication to access.

ephemeris.sleep.galaxy_wait(galaxy_url, timeout, verbose=False)

ephemeris.sleep.main()

Main function

ephemeris.workflow_install module

Tool to install workflows on a Galaxy instance.

ephemeris.workflow_install.import_workflow(gi, path, publish_wf=False)

Given a connection to a Galaxy Instance (gi) and a path to a Galaxy workflow file, this function will import the worklfow into Galaxy.

ephemeris.workflow_install.main()

This script uses bioblend to import .ga workflow files into a running instance of Galaxy

Module contents

ephemeris.check_url(url, log=None)

ephemeris.dump_to_yaml_file(content, file_name)

Dump YAML-compatible content to file_name.

ephemeris.get_galaxy_connection(args, file=None, log=None, login_required=True)

Return a Galaxy connection, given a user or an API key. If not given gets the arguments from the file. If either is missing raise ValueError.

ephemeris.load_yaml_file(filename)

Load YAML from the tool_list_file and return a dict with the content.