Assorted Notebooks

Notebooks!

Jupyter notebooks are a great way to transfer ideas into code and vice versa. Output is displayed on screen, with in line comments. Setting up the required python settings however might proove a little daunting since despite there many ways do do things not all of them agree. Instructions for setting up an Ipython notebook on lxplus (for CERN users) see below. <\p>

An alternative for CERN users is the SWAN service. SWAN (Service for Web based ANalysis) is a platform to perform interactive data analysis in the cloud. It allows those people with access to launch Python, cint or R notebooks with fully functional ROOT bindings in the cloud. All I have to do is log in using my web browser and the service handles the rest. Nominally any notebook that works using the tunnel interface also works with SWAN but not vice versa since SWAN includes some nice features like JSROOT javascript interactive plots (very useful for evaluating several functions at some point in the graph and even rendered in the html for you to play with). Some examples are given in both formats, otherwise only the SWAN interface is given.

Feynman Diagrams

I have explored two main methods for generating feynman diagrams. The first uses GkAntonius' Libraries. The second is the more common pyfeyn. This second method calls on latex libraries to create a pfd latex image which I have not been able to make agree with the notebook format. On SWAN the call to latex fails after 60s and localy processing the options does not work. In many ways this is prettier however much more difficult to display inline. Commonly I make all my images in SWAN using the GkAntonius method and only if I really need something extra pretty do I use pyfeyn.

• Higgs Feynman Diagrams, including EFT and LFV

RooFit!

Work with ATLAS always points back to ROOT and therefore machine learning is done with TMVA and statistics is done with RooFit. ROOT has recently made an effort to expand the accessibility of pythonic ROOT and has even partnered up with Jupyter to make pythonic ROOT available in notebook form. However I still rarely see python used for handling RooFit and so I made some notebooks to try and demonstrate the power of RooFit by translating some of the sections from the users manual into notebooks.

I found some RooFit and RooStats tutorials which I quickly converted for demonstration. I use the Kaggle Higgs Machine Learning data set fairly often when demonstrating statistical techniques. A long list of RooFit and RooStats examples have been converted to explore and use this data set. This section also features a tutorial on the use of HistFactory which I struggled to find otherwise. Histfactory allows the usage of ROOT objects such as TH1* histograms to be directly inputed into a statistical model for quick and efficient workspace building. If you intend to use systematic variations in your experiment, use HistFactory!

• Basics from the Manual
• Getting Started With RooFit (SWAN Version)
• Signal and Background-Composite Models (SWAN Version)
• Worked tutorials.
  • RooFit. Getting started and an introduction to workspace/statistical model building.
  • RooStats. Working with models and performing tatistical tests.
• Examples using Kaggle Machine Learning Higgs data.
  • Making Workspaces
  • Kaggle workspace building. Simple workspace building from data.
  • Kaggle HistFactory Example. Better workspace building.
  • Fitting
  • Kaggle fitting functions. Estimating parameters from functions.
  • Kaggle fitting templates. Estimating parameters from user defined PDFs.
  • Statistical Tests
  • Kaggle Profile Likelihood test.
  • Kaggle Baysian Markov Chain Monte Carlo calculation.
  • Kaggle Baysian Numerical Calculation.
  • Kaggle Upper Limits.
  • Kaggle Hypothesis Testing
  • Kaggle limits with CLs method.
  • Systematic Tests
  • Profiling individual NPs.
  • Pull Rankings
  • Ploting workspace contents (very useful ;-) )

At some point I will fill in the rest of the description to make a fully explained and described set of tutorials. Any annotations that exist thus far are liable to mistakes on my part so please contact me if you spot any mistakes or would like to help complete these explainations.

Methods for Multivariate Analysis with Theano

...Coming soon...

Accessing LXPLUS (Or any remote server) via SSH

Sure Jupyter Notebooks are great but I need the install system that I have set up on my institute cluster/lxplus? Never fear. Though lxplus doesn't have the most up to date jupyter version installed it's perfectly possible to open up a tunnel to the cluster, start a notebook on that system and run in your web browser.

Again, there are many ways to do this. This is the only way I found to do this on lxplus. If these instructions become obsolete I will try to change this page.

1. First ssh in to the remote server (for CERN users I'm just going to call this lxplus for now).
• ssh -L 7000:localhost:6000 username@lxplus.cern.ch
• this tells ssh to form the tunnel between lxplus and localhost port 7000
• the port numbers can be any 4 digit numbers. Have fun.
2. Set up password for the notebook.app
• bash-4.1$ ipython
• In[1]: from Ipython.lib import passwd
• In[2]: passwd()
• Enter password:
• Verify password:
• Out[2]: 'some:numbersandletters:morenumbersandletters'
3. Create Profile
• bash-4.1$ ipython profile create
4. edit ~/.config/ipython/profile_default/ipython_notebook_config.py
• # Password to use for web authentication
• c = get_config()
• c.NotebookApp.password=u'some:numbersandletters:morenumbersandletters'
5. Start notebook on port listed above
• bash-4.1$ ipython notebook --no-browser --port=6000
6. Now in your web browser of choice open http://localhost:7000
7. Enjoy