Command-Line Functions
Many of the functions of followthemoney can be used interactively or in scripts via the command line. Please first refer to the Aleph documentation for an intro to the ftm
utility.
Key to understanding the ftm
tool is the notion of streams: entities can be transferred between programs and processing steps as a series of JSON objects, one per line. This notion is supported by the related alephclient command, which can serve as a source, and a sink for entity streams, backed by the Aleph API.
Examples
The command line sequence below uses shell pipes to a) map data into entities from a database, b) apply a namespace to the entity IDs, c) aggregate entity fragments created by the mapping, and d) export the resulting entity stream into a sequence of CYPHER statements that can be executed on a Neo4J database to generate a property graph:
ftm map companies_from_db.yml | \
ftm sign -s my_namespace | \
ftm aggregate | \
ftm export-cypher -o graph.cypher
Here’s another example that fetches pre-generated entities from a URL and loads them into a local Aleph instance:
export URL=https://public.data.occrp.org/datasets/icij/panama_papers.ijson
curl -s $URL | \
ftm validate | \
alephclient write-entities -f icij_panama_papers
Reference
Please refer to the output of ftm --help
for a detailed reference of the ftm
CLI:
Usage: ftm [OPTIONS] COMMAND [ARGS]...
Utility for FollowTheMoney graph data
Options:
--help Show this message and exit.
Commands:
aggregate Aggregate multiple fragments of entities
dump-model Export the current schema model
export-csv Export to CSV
export-cypher Export to Cypher script
export-excel Export to Excel
export-gexf Export to GEXF (Gephi) format
export-neo4j-bulk Export to Neo4J bulk import
export-rdf Export to RDF NTriples
import-vis Load a .VIS file and get entities
map Execute a mapping file and emit objects
map-csv Map CSV data from stdin and emit objects
pretty Format a stream of entities to make it readable
sieve Filter out parts of entities.
sign Apply a HMAC signature to entity IDs
sorted-aggregate Aggregate sorted fragments of entities
validate Re-parse and validate the given data