(This blog post was originally posted on Bouke’s personal blog. Bouke is in Japan for an internship with the KamLAND-Zen collaboration at Tohoku University in Sendai, Japan. Read more about his trip in the Nikhef news article)
17:00 o’clock in Mozumi, Gifu prefecture. As I finish filling in a checksheet for the data acquisition (DAQ) systems of KamLAND-Zen, the small mountain village which I reside in — otherwise serene and quiet — suddenly brisks up: a speaker casts a cheerful tune unto the town and signals the end of the day. Having spent the preponderance of the morning and afternoon working within Kamioka mine, the happy melody is a welcome sound, for only the music of hissing pressure valves and clanking diaphragm pumps accompany you inside. But that is not to say that work within the tunnels is uneventful. Quite the contrary!
Over the past few days I have been trained by Hachiya-san (an RCNS Phd working on solar neutrino analysis) in order to be able to take over his hardware shift at the KamLAND-Zen detector site during the start of this week. It took me a while to familiarize myself with all the controls and check-ups involved, but with a bit of time and with Hachiya-san’s guidance, I managed to get the hang of it by the time my training ended. Let me guide you through the activities which comprise a regular KamLAND-shift day below.
Somewhat silly but essential nonetheless, one of the most important things to do when taking a shift, is to set a proper alarm. Shift-takers usually leave for the Kamioka mine together at 07:30 in the morning using a special university-owned car. So it’s important to stand ready near the parking lot, awake and well-dressed, when the time arrives. This is especially true during the current data acquisition period.
Since the collaboration’s working on the delicate issue of controlling the Xenon levels within KamLAND-Zen’s inner balloon at the moment, at least two experts need to be present within the mine at every hour. In practice this means that some professors and phd students have to spend several long consecutive nights concentratedly staring at the Xenon-filling displays from within the mine. In order to relieve them from their eight-hour duty and to allow them to get some sleep, two other experts arrive at the facility around 07:45 in the morning, taking the regular shift-takers with them.
So what does my day during a shift look like? I usually set my alarm around 06:25, after which I stumble to the dormitory’s kitchen to eat some breakfast: some European cereals with milk or a more traditional 卵かけご飯 (tamago kake gohan, cooked rice mixed with raw egg and soy sauce). Afterwards, I go back to my room and prepare for departure. There are three things that need to be done prior to leaving the dormitory. First off is a quick look at the KamLAND detector systems via the online monitoring system, to see if any trouble occured during the previous night. There is a small computer room opposite the dormitory’s kitchen, where one device is always running the monitoring display. After the check is done, I turn to the corner of the desk at the end of the room, to retrieve a portable handheld system (PHS) used for calls and notifications inside the mine. Next up, I check whether I took something for lunch. Since there’s a small watercooker and a microwave within the mine’s control room, we usually bring along either some cup noodles — the number of different kinds you can buy here is mind-baffling — or wrap some cooked rice in a piece of cling film to eat alongside a package of curry. Afterwards I go downstairs to change into a pair of boots. Having put on my personal safety helmet and having pulled a flashlight from the nearby powersocket, I’m finally ready to leave.
The 15 minutes drive from the Mozumi dormitory to Kamioka mine is a scenic one. A curvy highway cuts through the mountains, following a small waterbasin in the valley below. On both sides of the road, autumn trees colour the slopes in vibrant yellows, reds and greens. And large stone 鳥居 (torii, literally: bird abode) mark the entrances to shinto shrines, scattered across the hills. For all its quietness, the Mozumi area truly offers a breathtaking window unto the rural areas of Japan. I hope that I will be able to discover more of it during the coming months.
During the first few days of my shift training, the entrance to the Kamioka mine was closed when we arrived. There are no official mineworkers inside the tunnels during the weekend, so the gates remain shut on Saturdays and Sundays to keep any uninformed adventurers out. Yoshida-san, an experienced KamLAND researcher taking the Xenon shift during the period of my training, would open them up. But not before Hachiya-san and I completed some important checks first.
Outside of the mine entrance, one can find a small hut which houses a power generator and several systems for GPS tracking. This system is important, since we need accurate timing for the synchronization of electronics within the mine and inside of the detector itself. In total, three checks are done. First off, we verify that the power generator inside the hut is still functioning. There is a pannel at the back, containing a current meter as well as two red light indicators, which light up if enough power is generated. Moreover, there is a separate cabinet on the other side of room, where a rubber band can be observed to move whenever the generator is in operation. Outside of the GPS-hut, we find another cupboard, containing the GPS-system’s UPS board as well as a dehumidifier which keeps the interior dry. Only after both of these systems have been checked and after the workings of the generator within the hut have been confirmed, do we enter the mine.
Reaching the KamLAND-Zen detector site from the main entrance, takes about a five to ten minutes drive through a broadened, 2 kilometer length section of the dark and damp mining tunnels. These days, the car is usually parked close to the Xenon control room, so that the night shifters who are present can pick up the car and leave easily. A sloped walkway, some 10 to 20 degrees in elevation, leads to the DAQ control room.
The first thing shifters do, when entering the room, is conducting a thorough check of all the running DAQ procedures via the online monitoring system. Multiple panels indicate a variety of relevant variables, such as the system’s recording time, data flow rates, alarm notifications and PMT hit rates, as well as temperatures, humidity levels and gas pressures for a variety of different detector compartments. Since this is all rather involved, I won’t explain it in detail here. One thing I would like to mention, however, is that KamLAND-Zen is installed with two separate data acquisition systems rather than one. This is rather remarkable, so I think it deserves a little extra attention.
The KamLAND experiment started out with a single DAQ system called KamFEE (for KamLAND Front-End Electronics), which was built around the purpose of optimizing observable antineutrino event rates from nuclear reactors throughout Japan. Although the system performed very well — achieving sampling rates that allowed for the determination of PMT pulse arrival times up to the sub-nanosecond level — it turned out that several types of events were extremely hard to distinguish. Highly energetic muons that entered the detector, would cause an overshoot in the baseline voltages of individual PMTs which could last for as long as 1 millisecond. This prevented the accurate tagging of subsequent spallation events. To put it bluntly, our inner photodetectors were momentarily blinded by high charge events.
To decrease the period of insensitivity experienced by each PMT after high charge events (i.e. to reduce their deadtime) a new data acquisition system, called MogDAQ, was installed in 2009, which contained an electronic circuit for baseline stabilization. Although the new system wasn’t able to rid the photodetectors of the issue entirely, results exceeded expectations, with deadtimes being reduced by up to a factor 1000. Currently MogDAQ is used for tagging muon-induced spallation events only, with the KamFEE electronics taking care of all regular DAQ processes. But improvements are actively being worked on, potentially allowing the new system to take over various tasks during future phases of the experiment.
Enough about the data acquisition electronics for now. Let us continue our shift-taking tour where I left off. When the online system checks are finished, we print out two checksheets that need to be filled in during a walk around the detector. As mentioned during new shifters’ instruction briefings, it’s important to see the various detector elements with your own eyes and search for any unusual sight, smell or sound. The first detector compartment, we usually subject to such inquiries, is the water purification area, which can be found on the road between the control room and the liquid scintillator (LS) purification sites. Besides looking for any significant water leakages, we write down the water levels coming from and going towards KamLAND’s outer water-Cherenkov detector and depressurize the present water filters. Concerning the latter task, I sometimes play with the thought of bringing an umbrella as water may splurt out several decimeters high, depending upon the gas build-up within the filter tanks.
After finishing all water-system checks, we usually continue along the driveway through which we entered the mine, where we soon encounter the first and second LS purification areas. In the first area, KamLAND liquid scintillator is pumped through a tank for purification via the so-called water extraction method. By inserting LS and pure water in countercurrent directions within the tank, several metal element contaminations, such as uranium (U), thorium (Th) and potassium (K) can be largely reduced. This has to do with their larger solubility in water. The metals dissolve, after which they are transported away via drainage pipes. When this is done, the LS is pumped towards the nitrogen purge tower, where Radon gas, Oxygen and any remaining water is removed. This is to prevent any large Radon-induced background events within the experiment and to diminish the effects of oxygen-induced liquid scintillator quenching1.
But the purification isn’t quite finished yet there. Since several metallic oxides or metallic complexes aren’t filtered out by the previous methods, a second purification system is installed within the mine, which uses fractional distillation to further reduce any contaminations. This may sound more complicated than it is. Since the KamLAND liquid scintillator comprises three different components, i.e. Pseudocumene, Decane and PPO, with boiling points at 169℃, 174.2℃ and 360℃ respectively, it becomes possible to separate each part individually simply by boiling up to the correct temperature. A mixing tank at the end of the distillation area, subsequently mixes each component in again to the right degree. Once distilled and remixed, the liquid scintillator undergoes nitrogen purging once more to remove as much remaining Radon and other noble gases as possible.
The checks we do within the purification areas are relatively simple. We usually start off with the control panel at the first purification site. Are the Oxygen levels okay? And did the system measure any flammable gas? If the answers are yes and no respectively, we continue. Otherwise we call an expert via the portable phone. The same holds for any of the other checks we do. Are the various liquid scintillator oil levels within the storage tanks okay and are all systems operating in auto mode like they should? In case of a yes, we move on. Otherwise we write down the problem and notify an expert.
When the purification sites have been properly walked around, checking for any sort of leakages or problems, only four large sites for the detector inspection remain. The first is located right outside of the distillation area. Since nitrogen purging forms an integral part of the purification systems, it is important to see whether the nitrogen reserves within the storage tanks are sufficient. Hence we check for abnormally high or low pressures.
The last three areas which require inspection are located above the KamLAND detector itself. To reach it, we walk back along the way we came, go past the control room and climb up another slope towards the high voltage (HV) room. Inside we find eleven VME-crates, which collect all digitized signals from the detector’s photomultiplier tubes. We check that all crates are turned on and functioning and subsequently write down the forces measured by several load cells attached to the strings which hold the inner balloon. These are indicated on a few rows of displays, next to several humidity and temperature meters.
After all checks are done, we enter the dome, situated directly above KamLAND’s outer detector. This area is designated as a (low-level) cleanroom environment, meaning that we need to change into a cleansuit whenever we go in. Only when we’ve covered as much our skin as possible, using a large white suit, a pair of gloves, a hair net and a small mask do we step through the door. It’s a little messy inside, with various buckets, diapers and tools for cleanup scattered around the entrance. But this is not without a reason. Since the dome roof stands in direct contact with the surrounding rock of Mount Ike, ground water sometimes seeps through. This would endanger some of the cables and electronics lying on the floor when left undealt with. Not surprisingly, one of the biggest and most strenuous tasks within the dome and during the walkaround check in general, is to mop up any the water leakages and replace absorption sheets and buckets for water collection where needed. This usually takes quite a bit of time: several locations are hard to reach, requiring you to crawl behind pipes or installations. There are also several meters which we need to check within the dome, but I think I’ve talked enough about pressures and temperatures and volumes, so I’ll skip the details for now.
The last area which needs to be checked, before the walkaround is finished, is the electronics hut (or E-hut). Shut off by two separate doors, this area is always kept in the dark dark. So it’s essential to bring a flashlight along. On the inside, we find a vast collection of electronics boards and computers, which are responsible for the initial stages of the DAQ process. Since the area needs to be kept as dry as possible, three dehumidifiers flank the rightmost wall. We check whether they are still functioning and empty the containers if any water was collected overnight. After inspecting several triggers and making sure that all boards are on, the walkaround check is finally concluded. We usually finish between 10:00 and 12:00. The rest of the day we sit in the control room, filling in e-logs around every two hours and solving any issues whenever they come up.
Except for a few alarms, my shift training went by very smoothly. Hachiya-san returned to Sendai on Monday, after which I was immediately tasked with training the next new shifter, called Stefano. Ironically, he is much older than me and has a lot more experience working in environments like the Kamioka mine than I do. But that provided for a good opportunity for me to learn as well. As one of the developers of the cryogenics in CUORE — a rival experiment in the Gran Sasso mine in Italy which looks for neutrinoless double beta-decay in large crystals of Tellurium — he was able to tell me about several experimental details and take away some misconceptions I had about purification systems in general. Perhaps most importantly in the end however, we generally had a lot of fun!
Dusk falls as Stefano, the Xenon-shifters and I drive back towards Mozumi. It’s a strange sensation to observe so little daylight during the week. But when I think of it, the veil of night illuminates the misty valleys in its own ways. Where the yellow dots of autumn trees retreat into the shadows, blacks and browns which herald winter move towards the front. I wonder how much more I’ll see the craggy landscapes change. Perhaps this is what inspired the haiku’s that Matsuo Basho is remembered by.
In December, I hope to be able to visit one of the locations close to Sendai, which the famous poet visited on his travels. There will be a new blog post about it in the coming weeks for sure. In the meantime, however, I still have some catching up to do writing down my experiences with regards to the other locations I’ve visited. I’ll try to update the blog once a week again once I return to Sendai.
Footnotes
1. Oxygen-molecules decrease the overall light-yield of a liquid scintillator as a consequence of the direct transferal of the LS solvents’ excitation energy towards the Oxygen atoms, which may undergo spin-allowed transitions from the singlet to the triplet state (see e.g. Christan Buck (2016) or Xiao,Hua-Lin and Deng,Jin-Shan (2009))