Installing and Hello World¶
pip install --user piper
pip install --user --upgrade piper
Now, to test pypiper, follow the commands in the
Hello, Pypiper! tutorial: just run these 3 lines of code and you’re running your first pypiper pipeline!
# Install the latest version of pypiper: pip install --user piper # download hello_pypiper.py wget https://raw.githubusercontent.com/databio/pypiper/master/example_pipelines/hello_pypiper.py # Run it: python hello_pypiper.py
The actual code, in
hello_pypiper.py is a very simple but complete pipeline:
#!/usr/bin/env python import pypiper outfolder = "hello_pypiper_results" # Choose a folder for your results # Create a PipelineManager, the workhorse of pypiper pm = pypiper.PipelineManager(name="hello_pypiper", outfolder=outfolder) # Timestamps to delineate pipeline sections are easy: pm.timestamp("Hello!") # Now build a command-line command however you like, and pass it to pm.run() target_file = "hello_pypiper_results/output.txt" cmd = "echo 'Hello, Pypiper!' > " + target_file pm.run(cmd, target_file) pm.stop_pipeline()
When you run it, you should see printed to screen some output like this:
---------------------------------------- ##### [Pipeline run code and environment:] * Command: `hello_pypiper.py` * Compute host: puma * Working dir: /home/nsheff * Outfolder: hello_pypiper_results/ * Pipeline started at: (08-24 12:34:34) elapsed:0:00:00 _TIME_ ##### [Version log:] * Python version: 2.7.12 * Pypiper dir: `/home/nsheff/.local/lib/python2.7/site-packages/pypiper` * Pypiper version: 0.6.0 * Pipeline dir: `/home/nsheff` * Pipeline version: None ##### [Arguments passed to pipeline:] ---------------------------------------- Change status from initializing to running Hello! (08-24 12:34:34) elapsed:0:00:00 _TIME_ Target to produce: `hello_pypiper_results/output.txt` > `echo 'Hello, Pypiper!' > hello_pypiper_results/output.txt` <pre> </pre> Process 2325 returned: (0). Elapsed: 0:00:00. Change status from running to completed > `Time` 0:00:00 hello_pypiper _RES_ > `Success` 08-24-12:34:34 hello_pypiper _RES_ ##### [Epilogue:] * Total elapsed time: 0:00:00 * Peak memory used: 0.0 GB * Pipeline completed at: (08-24 12:34:34) elapsed:0:00:00 _TIME_ Pypiper terminating spawned child process 2318... child process terminated
This output is printed to your screen and also recorded in a log file (called
hello_pypiper_log.md). There are a few other outputs from the pipeline as well. All results are placed in a folder called
hello_pypiper_results. Navigate to that folder to observe the output of the pipeline, which will include these files:
These files are explained in more detail in the reference section outputs explained.
What’s next? That depends on if you’re interested in just running pypiper pipelines, or if you want to develop pypiper pipelines. The next sections are a series of HOW-TO articles that address each of these scenarios.