minsci.xmu.containers package

Submodules

minsci.xmu.containers.auditrecord module

Subclass of DeepDict with methods specific to eaudits

class minsci.xmu.containers.auditrecord.AuditRecord(*args)

Bases: minsci.xmu.containers.xmurecord.XMuRecord

Contains methods for reading data from EMu XML exports

parse_changes(whitelist=None, blacklist=None)

Parse values in the old and new values table in Audits

Parameters:rec (xmu.DeepDict) – data from eaudits Returns:List of named tuples containing the old and new values

parse_field(field)

Parse values found in a single field in the old/new table

class minsci.xmu.containers.auditrecord.Change(field, old, new)

Bases: tuple

field

Alias for field number 0

new

Alias for field number 2

old

Alias for field number 1

minsci.xmu.containers.bibliorecord module

Subclass of XMuRecord with methods specific to emultimedia

class minsci.xmu.containers.bibliorecord.BiblioRecord(*args, **kwargs)

Bases: minsci.xmu.containers.xmurecord.XMuRecord

Subclass of XMuRecord with methods specific to ebibliography

static clean_reference(ref)

Cleans punctuation, etc. left behind when ref has empty fields FIXME: This is horrible

static format_name(last, first, middle, use_initials=True, mask='{last}, {first} {middle}')

Formats a name according to the given mask

format_reference()

Formats reference according to publication type

get_authors()

Formats a list of author names

get_prefix()

Get prefix based on record type or keys

get_source()

Determines the type of the source/parent publication

is_biblio()

Checks if record is a reference

minsci.xmu.containers.mediarecord module

Subclass of XMuRecord with methods specific to emultimedia

class minsci.xmu.containers.mediarecord.EmbedFromEMu(*args, **kwargs)

Bases: minsci.xmu.tools.multimedia.embedder.Embedder

Tools to embed metadata into a file based on existing data in EMu

get_caption(rec)

Placeholder function returning the caption

get_copyright(rec)

Placeholder function returning copyright info

get_creator(rec)

Placeholder function returning the creator

get_credit_line(rec)

Returns short credit line

get_date_created(rec)

Placeholder function returning the date created

get_datetime_created(rec)

Placeholder function returning the full date and time created

get_headline(rec)

Placeholder function returning the headline

get_inventory_numbers(rec)

Returns a list of catalog numbers

get_job_id(rec)

Returns the import identifier

get_keywords(rec)

Returns a list of keywords

get_media_topics(rec)

Returns relevant media topics

get_object_name(rec, mask='include_code')

Returns the photo identifier or list of pictured objects

get_object_numbers(rec)

Returns list of catalog numbers

get_object_sources(rec, source='NMNH-Smithsonian Institution')

Returns list with museum name

get_object_titles(rec)

Returns list of object titles

get_object_urls(rec)

Returns list of object URLs

static get_objects(rec, field='MulTitle')

Returns list of catalog numbers parsed from MulTitle

get_source(rec)

Returns source of the multimedia file

get_special_instructions(rec)

Returns long credit line for special instructions

get_subjects(rec)

Returns media topics for this record

get_time_created(rec)

Placeholder function returning the time created

get_transmission_reference(rec)

Returns the import identifier

set_job_id(job_id)

Sets job id manually for images not imported into EMu yet

class minsci.xmu.containers.mediarecord.MediaFile(irn, filename, path, hash, size, width, height, is_image, row)

Bases: tuple

filename

Alias for field number 1

hash

Alias for field number 3

height

Alias for field number 6

irn

Alias for field number 0

is_image

Alias for field number 7

path

Alias for field number 2

row

Alias for field number 8

size

Alias for field number 4

width

Alias for field number 5

class minsci.xmu.containers.mediarecord.MediaRecord(*args)

Bases: minsci.xmu.containers.xmurecord.XMuRecord

Subclass of XMuRecord with methods specific to emultimedia

add_cataloger(cataloger)

Add a Cataloger instance to the MediaRecord

add_embedder(embedder, **kwargs)

Create an Embedder instance for the MediaRecord

check_filename(primary=True)

Verifies that filename follows best practices

copy_to(path, overwrite=False, verify_image=False)

Copies the primary file to a new location

Parameters:

• path (str) – the directory to copy the image to
• overwrite (bool) – specifies whether to overwrite existing file
• verify_master (bool) – specifies whether to verify copied file

embed_metadata(verify_image=True)

Updates metadata in the primary and supplementary images

fix_filename(fn=None)

Fixes filename to conform with best practices

get_all_media()

Gets the filepaths for all media in this record

get_catalog_numbers(field='MulTitle', **kwargs)

Find catalog numbers in the given field

get_photo_numbers()

Gets the photo number

get_primary()

Gets properties for the primary asset

get_supplementary()

Gets supplementary assets and their basic properites

match(ignore_suffix=False)

Returns list of catalog objects matching data in MulTitle

match_and_fill(strict=True)

Updates record if unique match in catalog found

match_one()

Returns a matching catalog object if exactly one match found

set_default(key)

set_filename(mask)

set_mask(key, mask)

smart_caption()

Derives image caption from catalog

smart_collections()

Populates DetCollectionName_tab based on catalog record

smart_keywords(whitelist=None)

Derives keywords from catalog

smart_note()

Updates note based on catalog record

smart_related()

Populates DetRelation_tab with info about matching catalog records

smart_title()

Derives image title from catalog

strip_derived()

Strips fields derived by EMu from the record

verify_import(images, strict=True, test=False)

Verifies import against path

verify_master(media=None)

Verifies download/copy of master file by comparing hashes

minsci.xmu.containers.minscirecord module

Subclass of XMuRecord with methods specific to Mineral Sciences

class minsci.xmu.containers.minscirecord.MinSciRecord(*args)

Bases: minsci.xmu.containers.xmurecord.XMuRecord

Subclass of XMuRecord with methods specific to Mineral Sciences

antmet = <_sre.SRE_Pattern object>

describe()

Derives a short description of the object suitable for a caption

geotree = None

get_age(pretty_print=True)

Gets geological age as string

get_catalog_number(include_code=True, include_div=False)

Returns the catalog number of the current object

get_catnum(include_code=True, include_div=False)

Returns the catalog number of the current object

get_classification(standardized=True)

Gets classification of object based on record

Parameters:standardized (bool) – if True, use GeoTaxa to try to group and standardize classification terms Returns:List of classification terms

get_collectors()

Gets all the collector’s field numbers for a record

get_field_numbers()

Gets all the collector’s field numbers for a record

get_guid(kind='EZID', allow_multiple=False)

Gets value from the GUID table for a given key

Parameters:

• kind (str) – name of GUID
• allow_multiple (bool) – if False, raises error if multiple values with same type are found

Returns:

First match from the GUID table for the key (if allow_multiple is False) or the full set of matches (if allow_multiple is True)

get_identifier(include_code=True, include_div=False, force_catnum=False)

Derives sample identifier based on record

Parameters:

• include_code (bool) – specifies whether to include museum code
• include_div (bool) – specifies whetehr to include division

Returns:

String of NMNH catalog number or Antarctic meteorite number

get_name(taxa=None, force_derived=False)

Derives object name based on record

Parameters:taxa (list) – list of taxa. Determined automatically if omitted. Returns:String with object name

get_political_geography()

Gets political geographic info for an object

Returns:List of place names in order of decreasing specificity

get_stratigraphy(pretty_print=True)

Gets stratigraphy as string

is_antarctic(metname=None)

Checks if record is an Antarctic meteorite based on regex pattern

summarize()

Derives and formats basic information about an object

minsci.xmu.containers.taxonrecord module

minsci.xmu.containers.xmurecord module

Subclass of DeepDict with methods specific to XMu

class minsci.xmu.containers.xmurecord.Row(irn, field, row, val)

Bases: tuple

field

Alias for field number 1

irn

Alias for field number 0

row

Alias for field number 2

val

Alias for field number 3

class minsci.xmu.containers.xmurecord.XMuRecord(*args)

Bases: minsci.dicts.deepdict.DeepDict

Contains methods for reading data from EMu XML exports

add(path, val, delim='|')

delete_row(key, i)

Deletes the row matching the given index

delete_rows(key, indexes=None, conditions=None)

Deletes any rows matching the given conditions from a table

expand(keep_empty=False)

Expands and verifies a flattened record

finalize(*args, **kwargs)

Runs any functions that require a carryover attribute

get_created_time(timezone_id='US/Eastern', mask=None)

Gets datetime of record creation

get_current_weight(decimal_places=2)

Gets the current weight of the object

Parameters:decimal_places (int) – the number of decimal places to which to round the weight Returns:Unicode-encoded string with the weight and unit, if any

get_date(date_from, date_to=None, date_format='%Y-%m-%d')

Returns dates and date ranges

Parameters:

• date_from (mixed) – path to date from field
• date_to (mixed) – path to date to field
• date_format (str) – formatting mask for date

Returns:

Date or date range as a string

get_datetime(date_from, date_to=None, date_modifier=None, time_from=None, time_to=None, time_modifier=None, conjunction=' to ', format='%Y%m%dT%H%M%S')

get_guid(kind='EZID', allow_multiple=False)

Gets value from the GUID table for a given key

Parameters:

• kind (str) – name of GUID
• allow_multiple (bool) – if False, raises error if multiple values with same type are found

Returns:

First match from the GUID table for the key (if allow_multiple is False) or the full set of matches (if allow_multiple is True)

get_location(current=False, keyword=None)

Returns the current or permanent location of a specimen

get_matching_rows(match, label_field, value_field)

Helper function to find rows in any table matching a kind/label

Parameters:

• match (str) – the name of the label to match
• label_field (str) – field in a table containing the label
• value_field (str) – field in a table containing the value

Returns:

List of values matching the match string

get_modified_time(timezone_id='US/Eastern', mask=None)

Gets datetime of last modification

get_notes(kind)

Return the note matching the given kind

get_paths(rec=None, path=None, paths=None)

get_reference(*args)

Returns a list of values corresponding to the table rows

Parameters:*args – the path to a value in the dictionary, with one component of that path per arg. Returns:If the last arg is a field (as opposed to a reference table), this function will return a list of values, one per row. If the last arg is a reference table, it will return a list of XMuRecords.

get_rows(*args)

Returns a list of values corresponding to the table rows

Parameters:*args – the path to a value in the dictionary, with one component of that path per arg Returns:List of values, one per row

get_table(*path)

Returns the table to which the field specified in path belongs

get_url()

Gets the ark link to this record

is_new(found)

Checks if current module:irn exists in found

Parameters:found (dict) – marks irns already found as True Returns:Boolean expressing if the current record has already been seen

This method can be invoked manually inside the XMu subclass when reading XML exports from a directory containing multiple, potentially overlapping record sets to prevent (a) the same record from being read twice or (b) an older version of a record from overwriting a more recent one.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D

simple_pull(path)

Returns data from path in DeepDict

Parameters:path (mixed) – the path to an EMu field as a string or list Returns:Value for the given path

smart_pull(*args, **kwargs)

Pull data from the record, formatting the result based on the path

Parameters:*args – the path to a value in the dictionary, with one component of that path per arg. If args[0] contains one or more dots, the path will be expanded from that and ignore subsequent args. Returns:An atomic field returns a string A reference pointing to a single field returns a string A simple table returns a list of values A reference table that specifies a field returns a list A reference table returns a list of XMuRecord objects A nested table returns a list of lists Return type:Value for the given path, formatted as follows

to_refine()

Maps EMu data to Google Refine

FIXME: Needs to be cleaned up and tested

unwrap()

Removes outermost level of XMuRecord

This simplifies the paths needed to pull data from the record. The record will need to be wrapped again before writing to XML.

Returns:Unwrapped XMuRecord. In a typical use case, this means the paths used to retrieve data do not need to include the module name.

verify()

wrap(module)

Wraps the XMuRecord with name of module

Parameters:module (str) – name of module to use as key Returns:Wrapped XMuRecord. In a typical use case, this means the paths used to retrieve data need to include the module name.

zip(*args)

Zips the set of lists, padding each list to the max length

minsci.xmu.containers.xmurecord.standardize(val)

Standardize the format of a value

Module contents

Defines module-specific containers for working with EMu data