Neutron Data Acquisition Steven Hartman BES Detector Workshop, August 2012
Outline Introduction Survey of neutron data acquisition systems Identified Needs R&D efforts underway at SNS Conclusion 2 Managed by UT-Battelle
Introduction: System level approach Data Acquisition and Instrument Control Interface to Detector Systems Hardware and software Interface to beam line systems (choppers, optics, etc.) Interface to sample environment equipment (temperature, pressure, magnetic field, etc.) Experiment automation, scripting, scanning User Interface Data Management Interface to Data Analysis and Visualization 3 Managed by UT-Battelle
Introduction: Functional Diagram Timing Digi0zer Acquire Data Store and Catalog Data Reduce Data Data Storage User Analyze and Visualize Data SE, motors, choppers Slow Controls Experiment Automa0on Publish and Curate Data 4 Managed by UT-Battelle
Introduction: Challenges Higher power pulsed sources, bigger detector arrays, time of flight information, event data Greater data rates Larger data sets More complex analysis needs Neutron user community Growing user base Newcomers to neutron scattering Need to maximize the value of the data produced 5 Managed by UT-Battelle
Survey: DAQ for Neutron Facilities 6 Managed by UT-Battelle
Survey: HFIR High Flux Isotope Reactor, ORNL, USA Reactor-based source SpICE (Spectrometer and Instrument Control Environment LabView Windows UI, instrument control, scanning/automation, analysis capability 7 Managed by UT-Battelle
Survey: ILL Institut Laue-Langevin, France Reactor-based source NOMAD Java UI C++ service CORBA Linux 8 Managed by UT-Battelle
Survey: SINQ Swiss Spallation Neutron Source, PSI, Switzerland Accelerator-based source (spallation) SICS and Gumtree SE Java UI (Eclipse RCP) SICS backend Linux 9 Managed by UT-Battelle
Survey: ISIS ISIS, Rutherford Appleton Laboratory, UK Accelerator-based source (spallation) LabView and OpenGenie Windows Software framework under review with prototype EPICS test stand under development 10 Managed by UT-Battelle
Survey: SNS Spallation Neutron Source, ORNL, USA Accelerator-based source (spallation) Custom software suite (currently) C++, Labview, Python, Windows Event mode data acquisition High data rates, will increase Upgrades and planning in process Operability and maintainability EPICS for instrument control and UI Improved data handling Better integration with data analysis 11 Managed by UT-Battelle
Survey: ESS European Spallation Source, Sweden Accelerator-based source (spallation) Anticipate operations beginning in 2019 Conducting reviews of current neutron source DAQ systems Data Challenges Up to 300 million events per second across the instrument suite Data archiving estimate of ~8 PByte per year 12 Managed by UT-Battelle
Survey: Summary Little commonality across neutron sources to date Detector interface electronics Often custom designed Data acquisition software Developed in-house Instrument control software Often developed in-house plus some commercial software User Interface No common look and feel 13 Managed by UT-Battelle
Survey: Options Initial efforts underway to find areas for collaboration across sources for instrument control and user interface SNS, ISIS, ESS,... EPICS as a framework Already in use for accelerator control and light-source beam line control Collaboration in data reduction/analysis now underway Mantid Project: ISIS, SNS 14 Managed by UT-Battelle
Needs 2011 ASCR/BES Data Workshop 2012 SNS/ESS Accelerating Data Analysis Workshop 15 Managed by UT-Battelle
Needs Data Reduction and Data Analysis in real time with Data Acquisition Reduce and analyze data as it is taken Data files available instantly at end of run 16 Managed by UT-Battelle
Needs Feedback from analysis to experiment control Integration of modeling, experiment and data analysis 17 Managed by UT-Battelle
Research and Development at SNS Electronics Electronics Timing and Accelerator Data DSP PreProcessor Aggregator SMS Streaming Translation Streaming Reduction Live View NeXus Data: PFS Sample Environment Equipment, Choppers, Etc. Slow Controls: EPICS Scan Service Experiment UI: CSS Analysis UI: Mantid 18 Managed by UT-Battelle
R&D: ADARA ADARA Accelerating Data Acquisition, Reduction and Analysis Collaboration NDAV (NScD), Tech Int. (NCCS), IDAC/RAD (NScD) Shared Resources NeXus Translation Services Per Beam Line Resources Detectors Stream Management Service Data Portals Aggregates event and environmental data Streams data to Translation Services and Data Reduction and Analysis Electronics Detector Preprocessor(s) (1 or more) User Interface Global High-performance file system Pushes Events + Timing Hosts analysis and environmental controls GUI(s) Slow Controls Sample Environment & Scanning Services Pushes environmental data Receives controls commands for processing and response Analysis Clusters 19 Managed by UT-Battelle Analysis Workstations Manages sample environment, choppers, motors, etc. Streaming Data Reduction & Analysis Services Receives aggregate event stream and provides live reduction & analysis services
R&D: Low-level Electronics Timing Receiver Bringing accelerator timing signal closer to the detector electronics Prototype system on commissioning beam line Beam monitors, chopper phase, fast synchronous signals Treat like a detector Stream as events Standardize interface boards FMC modules 20 Managed by UT-Battelle
R&D: Low-level Electronics TCP/IP as data bus Need to stream data over network anyway Technology moving in this direction Replace custom hardware with commodity components Working on conceptual design for UDP from FPGA to computer How far down? FPGA vendors pushing network capabilities 21 Managed by UT-Battelle
R&D: Slow Controls and UI EPICS Toolkit In use for accelerator controls at SNS and light-sources Used for beam line controls at a number of light sources Prototype using HFIR Imaging beam line Simulated images and motors at this point Uses devices drivers previously developed for synchrotron beam lines 22 Managed by UT-Battelle
Conclusions Look at the whole data flow: acquisition, instrument control, reduction, analysis, visualization, data storage, access Systems engineering approach Optimize the whole data flow and the whole work flow Be aware of maintenance and support effort Software development is not free Effort required for quality software often underestimated Look for opportunities for collaboration We are experimental facilities for Users to: Collect the most (appropriate) data during their beam time Understand that data as much as possible/practical 23 Managed by UT-Battelle
Questions and Comments 24 Managed by UT-Battelle