Present:

R. Bock, R. Cranfield (chair), S. Gonzalez, N. Ellis, Y. Ermoline, B. Green, M. Huet, M. Joos, I. Mandjavidze, M. Mueller, J. Peterson, B. Pope, R. Scholte, S. Tapprogge, E. van der Bij, J. Vermeulen, F. Wickens

Notes by R. Cranfield and J. Vermeulen

Update on parameters (J. Vermeulen) (Slides)

Some event fragment sizes have been reduced. N. Ellis: TRT low luminosity event fragment size will come down by a factor of at least 2 if zero suppression is applied in the ROBs (will pass info to Jos). Updated numbers for the maximum data volumes (output per ROBin) for LVL2, EB and total were shown. The DAQ approach (requesting of data not driven by sequential processing) was also modelled and found to result in much higher total output rates  and data volumes per ROBin (see slides). R. Bock noticed that this is only a local traffic requirement, N. Ellis: but they are important numbers in agreeing architecture. I. Mandjavidze: are the "copies" copies or pointer-copies (from ROB to LVL2 interface) in the DAQ view ? J. Peterson: the DAQ-1 "view" assumed in the discussion is only one implementation option. N. Ellis: it is important that the numbers and arguments are documented for discussion.

Estimates for required buffer sizes were also shown with and without "fragmentation" (i.e. efficient re-use of holes in circular buffer) (numbers based on very long LVL2 latencies for low luminosity which is due mainly to full-scan (which might be done by a local pre-processor) ).

Performance measurements (see ROB Complex Web page for descriptions)

UK ( B. Green)

ROB-in performance meaurements: the test setup was described, the results show that the original design specification is achieved (100kHz @ 6% requests), the memory bandwidth is a limiting factor at high enough rates. The software was not optimised for performance. Also processor & memory speeds could now be increased. The results show that the performance is in the right ball-park.

R. Cranfield: main ROB Complex testbed is DAQ-1 prototype - report still in draft, it is hopefully available before Beatenberg.

Saclay (I. Mandjavidze)

ROBIN: the mk1 version without input section is used, the mk2 version  (with input section and 100 MHz processor in stead of 33 MHZ processor) should be available before the end of the year. A ROB complex with chained output directly to the network interface card is studied. Message rate plots were shown for data requests & event clear requests.

NIKHEF (J. Vermeulen)

An important design point is the "paged fifo" for fragment info. To try high input rates the paged fifo is emulated by the SHARC (checked against real S-link input). Plots with measurement results were shown and the results were fitted to a simple formula; the formula was used to extrapolate to other parameter values. Latency histograms show separated peaks due to characteristics of polling loop. A ROB Complex with 2 CRUSHes can be used to emulate a 4-input complex (2 outputs per ROBin). Output is via a PCI bus with a  SHARC connected to a PCI interface. The results show that for a certain RoI request fraction the maximum event frequency reduces with number of ROBins. PCI output is currently at half theoretical - need to investigate further when PCI analyser is repaired!

Mannheim (M. Mueller) (slides)

As results for ATLANTIS are not yet available the old measurements were referred to: PCI output: 7 kHz with 4k fragments, 4.1kHz with 4k + pre-processing. Limitations are due to 20 MHz access of RAM (dual-port emulation) & PCI, therefore ATLANTIS is the next step. The basic idea is to combine several input links before PCI: 4 FPGA based ROBins on one board implemented with 2 Xylinx Virtex 600 FPGAs, with for each ROBIn 4 MByte SRAM buffer memory and 128 kByte dual-ported RAM. Input is via MLINK: half S-link size card with LDVS chips controlled by external FPGA i.e. S-link functionality is split between card & FPGA. E. van der Bij: what is the transfer speed via PCI? M . Mueller : 132 MB/s with PLX chip (32-bits, 33 MHz), E. van der Bij: an FPGA-based PCI bridge could be used. M. Mueller: using another bridge would be difficult.

MFCC (J. Peterson)

The use of the MFCC (PMC boar from CES with FPGA and PowerPC processor) as ROBIn is studied by the DAQ-1 group. It fits well into the infrastructure of DAQ-1 in view of the possibility to run LYNXOS on the PowerPC processor. S-link data can be passed at 140 MByte/s via the PowerPC bus .

Discussion on further bencmark measuerements before Beatenberg

M. Mueller : no further measurments, J. Vermeulen : some additional measurements, Saclay : waiting for mk2, R. Cranfield : DAQ-1 results should be available in Beatenberg. J. Vermeulen is to circulate the specification of the measurements again.

Completion of Master Working Document (Cranfield)

The MWD ("Options for the ROB Complex") is the main deliverable. Timescale is tight. Need to identify people to a) provide new input, b) check sub-sections.

Layout

The UML description is to be incorporated in Chapter5 (Context), Scenarios & Measurements chapters are to be added (8 & 9).

For scenarios: propose to put the existing documents into own chapter (with some trimming) and to summarise in the "Implementation" chapter. Jos will try to also provide an introductory section for the scenarios chapter covering the modelling of grouping options (Active ROB Complex). For the measurements chapter it was agreed to follow the same approach. Prototype descriptions for some systems are needed, and all input is needed in a format that could go into the MWD. Also a table of comparitive results need to be drawn up (for Beatenberg also).

There were no comments on chapters 1-7.1 (apart from UML).

Summary of work for other sub-sections of chapter 7 (R. Cranfield to circulate people responsible):

7.2/7.3/7.4 : L. Levinson providing input (awaiting ROD feedback). Review group: Ellis, Wickens, McLaren, LeDu. M. Huet provides text on how the ROBs know where to send accepted data. N. Ellis :  LVL1 Id needs to be extended, to be dealt with by DIG.
7.5 : Data Profile : the modelling results need to be updated
7.6 : Level-2 Strategy : review group: Bock, Bystricky, Clarke, Ellis, Wickens
7.7 : Data Format : review group: Mandjavidze, Muller
7.8 : Operational Framework : review group: Huet, Bogaerts?, Bob Jones?
7.9.1/7.9.2/7.9.3 : Cranfield/Vermeulen to reword. Review group: Bock, Muller, Mandjavidze.
7.9.4-7.9.? : Input awaited from "technical group": van der Bij, McLaren, Kugel, Green, Mandjavidze, Jansweijer.
7.10 : Software : input re Internal APIs from Huet &  Cranfield; input re software features from: Huet, Vermeulen, Mueller, Cranfield, Ermolin? review group: same.
7.11 : Testing : review group: "technical group": van der Bij, McLaren, Kugel, Green, Mandjavidze, Jansweijer

Status reports

E. van der Bij

The PCB layout of a new S-link card and a new optical transceiver and connector (G-link) were shown. The S-link card allows to run a ReadOut Link at full speed (160 MByte/s) using 2 optical link pairs (current cost: 1 Kchf per link for one-offs), or at a speed of 128 MByte/s if one link pair is used.

R. Bock

One 4-processor (550 MHz Xeon-III) 2-PCI mini-SMP system with 512 MByte of shared memory is now available. NT is running, the system is on its way to Mannheim. The API is defined. 7 slots (& microEnables) are available.

The modelling exercise has been documented. Preprocessing helps for reducing bandwidth, but the data bandwidth to the EB becomes a problem. Feature extraction looks less promising (because complete RoIs are not containedwithin a complex), though it could still be done with partial use of network. Results are expected to be available by Beatenberg.

Mannheim (M. Mueller)

ATLANTIS IO board ready by end of year? Most of the manpower is on ATLANTIS computing board (as coprocessor) for full scan TRT.

NIKHEF (J. Vermeulen)

Will continue with measurements. Also plan to use 2 newly purchased Gigabit Ethernet interfaces. Scenario 2 (ROB complex output via S-link to farm processors) will also be explored. Also doing hardware development for muon precision chamber RODs using CRUSH module.

Saclay (M. Huet)

Starting pcb design for ROBin mk2.

UK (Cranfield/Green)

Continuing support of DAQ-1 & possibly more ROB-in performance measurements. Prototyping of simple ROB by combining UK ROB-in & fast ethernet card (using CERN simple protocol).

UML description (M. Huet) (slides)

Top-down approach (starting from URD). High-level description. Discrepancies between LVL2 & DAQ views were illustrated - pros & cons were listed. Agreement on high-level design in LVL2 & joint description with DAQ is needed. Chapters 5 and 6 of the document will be removed as they are too detailed for a high-level discussion with the DAQ group. 15 November outptut shouldbe available for the Steering Group meeting of 17 November.

Review of goals & state of documentation (discussion)
 

Preparation for Beatenberg (discussion)

Need for objective set of points around discussion of discrepancies between LVL2 & DAQ views. The Pilot Project ROB presentation at Beatenberg will only be 30-40 minutes. A rough outline of this presentation was agreed: context; project planning; requirements; measurements/feasibility; scenarios for full system. The UML analysis was thought to be best discussed in the common sesssion.

N. Ellis : pre-processing should also be mentioned.

R. Cranfield : work after the TP : one ROB group is needed, UK intends to look to "simple ROB" scenario. N. Ellis : a larger ROB system demonstrated in a testbed is needed. M. Huet : distribution of work should be agreed, N. Ellis : clear work program needed, tasks should be divided up, the recommendation of the commission is important. A discussion on whether cost should be addressed in the MWD did not lead to a conclusion.