SLD: S-Link Driver for Linux.
At Nikhef investigations are in progress for the Atlas project (check their website for information). One of the topics is S-Link, which is to be used to pump data from one system (perhaps a detector) to a fast computer (to process the data). One possibility to connect an S-Link LDC (Link Destination Card) to a generic PC is to use the CERN SSPCI interface. This driver works only with that interface. The assignment indicated the following requirements (among others):
From a few conversations with people, it was clear that this driver
would not be used in a generic-use Linux environment, which allowed to
take decisions to be made different from the generic-use case. One of the
first decisions in this manner was to avoid system calls, since they are
expensive with regard to CPU power.
lmbench showed that a simple system call costs 0.5814 microseconds (on a 700Mhz PentiumIII), so at the expected datarate of 60MB/s with 1.5KB per event and 3 system calls per event (2 controlwords and 1 block datawords) this would mean 40960 events, or a whopping 122880 system calls per second. This would require about 71.4 milliseconds already for the simple system call.
The SSPCI interface uses the AMCC s5933 Matchmaker chip. The documentation is available from the AMCC website. AMCC only has the errata for the QE version available. Here are also the QB and QC errata and a letter about the transistion to QE, which are not available from the AMCC website (duh!).
The driver has been developed as a kernel module and uses a dynamic major number. As long as you have only one driver using dynamic majors, or always load drivers in the same order, the driver will get the same major number.
There are two configurable items on this driver and those are the buffer size and the number of buffers that are used. The lower limit on these two is 128 bytes and 10, respectively (just to make sure that the driver will function somewhat) and the upper limit is PAGE_SIZE for the buffer size (which is 4KB on Intel systems) and unlimited for the number of buffers (in the driver anyway!).
Internally the driver maintains a circular queue with databuffers and
it fills the buffers at the head of the queue. In user space some program
takes a buffer from the tail of the queue, processes the data in it and
frees it again. Some code from the driver needs to be compiled with the
program to access this interface. The basic operation of the user program
looks like this (for clarity error checking of select() was removed):
Sldtest offers the options to display S-Link data and control words,
driver informational messages, statistics and various other things. It
uses the driver the way any other application would use the driver, so
it's a good example when you're developing your own application. The other
program is sldstat, a program like iostat and vmstat. It displays one line
of information at a time with the most basic statistical information from
the driver. A program in this style only has the ioctl interface to perform
any useful work. This allows for control of the return lines and getting
the status of the S-Link and statistical information.