S-link drivers for Windows NT

Nata Kruszynska

NIKHEF, November 1997

Hardware:

Simple S-link PCI devices are based on the AMCC S9533 PCI interface and their purpose is to provide a uni-directional dedicated connection, sending big chunks of data from the source device on one PC to a destination device on another PC. The S-link specification defines four additional return lines. Data can be sent via these lines by writing to OMB1 of the AMCC controller and can be received by reading IMB4 byte 3 on another sice of the S-link connection. The formats and line assignment differ between source and destination. Both devices are using only AMCC registers mapped into single memory region.

Current implementation:

There are two drivers - source slink and destination slink. Each can cover more than one "minor" device and can control more than one S-link device. The transfer of data is uni-directional. This means that the source driver supports writing data and returns error on an attempt to read data, whereas the destination driver can read, but can not write. The drivers are kept as simple as possible. Both establish an AMCC driver for a device for which the AMCC chip is memory-mapped in a single memory region. The part of the required functionality not implemented in drivers is left to a supporting library and to the user's programs on both sides of the S-link connection. In the future, a higher level driver could be provided, when experience with the current implementation makes this desirable.

General features:

Both drivers use exclusive access of each of the installed "minors". This feature could be replaced by semaphoring, but we do not think that that will be really useful for S-link.
Both drivers are implemented as DIRECT_IO_ACCESS devices. I.e they keep the user's read/write buffer to be used for transfers locked in the memory without copying the data. This feature is NT specific and can not be easily used on another system.
Both drivers use DMA packaged AMCC master mode driven by the i/o done ( count = 0) interrupt from AMCC. I.e. they pick physically contiguous chunks of the locked user buffer and perform an AMCC DMA (FIFO) operation on it. On i/o done they look if the next DMA operation is needed. If that is so, the next DMA is initiated and i/o done awaited, otherwise i/o is completed.
On both drivers every AMCC works in PCI-master mode. It means that the implementation of receiver buffers, checking the validity of the data etc. does not belong to the scope of the driver, being left to the user software or later-on to the higher level S-link drivers on both sides.
Ioctls are supported to access AMCC registers. S-link specific routines can be implemented on the basis of OMB1-IMB3 accesses.
The user program can be notified by an "Event" on changes in the S-link control lines caused by the counterpart's access. This notification is voluntary. It is left to the user to read via ioctl the IMB4 register and decode the S-link lines setting from it.
The user can also use IMB4 to synchronize source application with the application on destination. After writing to the destination the source application can use ioctl writing OMB1 ( setting s->d lines) to force an IMB4 interrupt on the destination. The IMB4 interrupt on the destination causes the pending i/o to be completed, whereas on the source it does the same only when the connection is reported to be down.
Both drivers are equipped with an i/o cancel routine which will unblock the driver after exit of the utility with the unfinished i/o .
No timeouts or retries are provided, this task is left to the higher level driver or to the user's protocol.
The drivers are equipped with extensive check-mode diagnostics. Free mode ( installation standard ) will keep only fatal errors and a few serious warnings which could impair the performance.