E3DDS Stakeholder 20191017

From neicext
Jump to navigation Jump to search

Time/Location

  • Meeting to be held in Tromso, NO on Thursday October 17th 2019, 10:00 - 16:00 CET


Objectives for the meeting

The E3DDS stakeholder forum gathers representatives from the NeIC EISCAT_3D Data Solutions project, NeIC, EISCAT, EISCAT_3D users, national e-Infrastructure providers and network providers in order to achieve the goals:

  1. To update each other about progress of projects and e-Infrastructure roadmaps;
  2. To intensify the collaboration between the EISCAT_3D Data Solutions project and the providers for network and e-Infrastructures.

The general objective of this meeting is to bring together the EISCAT_3D technical community with national providers of HTC/HPC and networking from the relevant countries. The goals of the meeting include the need to update each other of statuses and time-lines of national providers and the EISCAT_3D activities.

Visit to the Skibotn site

  • The site looks like this.
  • Visit inside the secure area not possible. (It's a sand/gravel pit right now).
  • On Wednesday October 16th, there is at least one vehicle going from Tromso to Skibotn for a visit. (Tomasz, John, Craig).
  • EISCAT people can borrow a vehicle from Ramfjordmoen site.
  • Local car (Dan Jonsson) will also go to Skibotn on Wednesday.
  • For possibilities for Friday October 18th, we will have some information in the next week.

Participant List

  • John White (NeIC)
  • Tomasz Malkiewicz (NeIC)
  • Ari Lukkarinen (CSC)
  • Bjorn Torkelsson (HPC2N)
  • Dan Jonsson (UiT)
  • Craig Heinselman (EISCAT)
  • Lars Fischer (NORDUnet)
  • Olaf Schjelderup (Uninett)
  • Harri Hellgren (EISCAT)
  • Thomas Ulich (SGO)
  • Carl-Fredrik Enell (EISCAT)
  • Yasunobu Ogawa (NIPR)
  • Roy Dragseth (UiT)
  • Ingrid Mann (UiT)
  • Juha Vierinen (UiT)

Agenda

  • Meeting chair: Tomasz Malkiewicz, NeIC
  • 10h00 - 10h30 Coffee
  • 10h30 - 10h45 Welcome slides (Tomasz Malkiewicz, NeIC)
  • 10h45 - 11h10 Status of the EISCAT_3D project slides (Harri Hellgren, EISCAT)
  • 11h10 - 11h35 Status of the E3DDS project slides (John White, NeIC)
  • 11h35 - 12h00 Status installation to Skibotn site slides (Olaf Schjelderup, Uninett)
  • 12h00 - 12h25 Nordic Network Plans slides (Lars Fischer, NORDUnet)
  • 12h30 - 13h15 Lunch
  • 13h15 - 13h30 Preparation for EISCAT_3D Karesuvanto site slides (Thomas Ulich, SGO)
  • 13h30 - 13h50 Progress on Norway National e-infrastructure slides (Roy Dragseth, UiT)
  • 13h50 - 14h10 Progress on Swedish National e-infrastructure slides (Bjorn Torkelsson, SNIC)
  • 14h10 - 14h30 Progress on Finnish National e-infrastructure slides (Teemu Kiviniemi/Ari Lukkarinen, FUNET)
  • 14h30 - 15h00 Coffee.
  • 15h00 - 15h15 Data and Metadata slides (Carl-Fredrik Enell , EISCAT)
  • 15h15 - 15h30 Report on On-site data processing simulation deliverable slides (Ari Lukkarinen, CSC)
  • 15h40 - 15h45 Wrap-up, next meeting, propose April 7th 2020 (Tomasz Malkiewicz, NeIC)

Minutes

Anyone who wants to drop comments:

https://docs.google.com/document/d/1NwD0MKHg2IIVPQCRkchI0FRWZa44ZS6JKeMa1-5pmtM/edit

  • 10h45 - 11h10 Status of the EISCAT_3D project slides (Harri Hellgren, EISCAT)

Question: What are the requirements for time synchronization on and between sites?

Answer: The WAN "can" do White Rabbit timing over long distance. But WR is mainly needed for the beamforming. The site to site measurements are essentially de-correlated. GPS timing may not be considered stable or reliable enough... drifts over hours.

Comment: Could you use a divide and conquer stratgey for the beamforming? i.e. multiple stages?

Comment: AMD/Intel performance: AMD CPU may not have raw performance of AMD but the data throughpu is much higher for AMD. e.g. One socket on the proposed test server "could" handle the data rate of the whole site.

  • 11h10 - 11h35 Status of the E3DDS project slides (John White, NeIC)

The beamforming input is based on 100 Gb/s Ethernet. The test server purchased by EISCAT is Dual socket with 64 cores AMD EPYC Rome with PCIe 4.

Questions: How about the data transfer to data centre? The location of the data centre?

Answer: The location of the Data Centre is not yet determined.

Comment: The data centre has to be located close to WAN ring.

Comment: EISCAT needs hands-on access to equipment during commissioning and early operation phase.

Comment: Access to computing servers is usually through laptp/termnial unless something has physically failed.

Comment: The first year(s) of data can be hosted in Skibotn site.

  • 11h35 - 12h00 Status installation to Skibotn site slides (Olaf Schjelderup, Uninett)

Question: The man-hole that contains the fibre optical terminations? Is this secured?

Answer: Yes. The man hole cover is locked.

Question: Are there connectors in the hole?

Answer: No. Normally the fibre optic cables are spliced from there onwards.

Question: If you want to add a new cable do you have to do the same digging as illustrated?

Answer: Should not have to. We have installed spare pipes with no fibre installed. Otherwise this is expensive to do afterwards!

  • 12h00 - 12h25 Nordic Network Plans slides (Lars Fischer, NORDUnet)

Question: What about data resilience? Can data be lost?

Answer: With such a topology, you are only moving photons along "both" paths. If there ia a problem on one path, then the other takes over.

Question: For an optical network, is there any issue to reconfigure? i.e. No problems to move equipment?

Answer: This is only moving a rack. Easier than moving the compute and disks if a data centre needs to be moved.

Question: Does this imply that a data centre can be located at any one of the sites? Skibotn etc?

Answer: With optical network... Yes.

Question: How about the fibre from Skibotn to FI? Does it cross the border by Kilpisjarvi?

Answer: The FUNET 2020 deployment brings fibre to the FI (Karosuvanto) site. FUNET has ordered this fibre service from the provider. The fibre is in the ground.

  • 12h30 - 13h15 Lunch
  • 13h15 - 13h30 Preparation for EISCAT_3D Karesuvanto site slides (Thomas Ulich, SGO)

Question: What are the three masts?

Answer: Calibration, lightning etc.

Question: Do you plan to have other observatory instruments at these sites? i.e. FI, SE etc?

Answer: We could do, optical? Possibly optical 3D system.

Comment: From the network people... if there are to be other instruments at a site then please think about the network requirements... with an optical Dense Wavelength Division Multiplexing (DWDM) system there is no real capacity limitations (apart from cost). But an experiment will appear to be at the data centre where the fibre connects to the internet. Then would have to be routed over "normal IP" to the final destination.

  • 13h30 - 13h50 Progress on Norway National e-infrastructure slides (Roy Dragseth, UiT)

Question: What is the NO contribution to the Lumi consortium?

Answer: 4/110 M-Euros. EU funds the other 100 M Euros. Therefore a direct approx 2%.

  • 13h50 - 14h10 Progress on Swedish National e-infrastructure slides (Bjorn Torkelsson, SNIC)

Comment: The SurfSARA network tests are for the upcoming SKA.

Question: Norway goes from 4 to 2 supercomputer centres for New machine and replacement. Are there plans for SE?

Answer: We are still finalizing for SNIC 2.0. So don't really know yet.

  • 14h10 - 14h30 Progress on Finnish National e-infrastructure slides (Teemu Kiviniemi/Ari Lukkarinen, FUNET)

Comment: The data sets will probably (in fact will according to FAIR) may be required to be provided as a copy to some sort of national archiving service in the future. Therefore will require identifiers such as Open Archival Information System (or OAIS).

Question: How about the Data preservation for EISCAT_3D?

Answer: This is not an impact to EISCAT but should store the data in well-defined model and metadata. Using such as PIDs.

Comment: For tape storage ... there is only one manufacturer any more. IBM. The price gets very expensive.

  • 14h30 - 15h00 Coffee.
  • 15h00 - 15h15 Data and Metadata slides (Carl-Fredrik Enell , EISCAT)

Comment: It is possible to go from raw voltage (level 0) to HDF5 files. This is being done now. No checksum or compression. Go for HDF5 format as soon as possible in the data chain.

  • 15h15 - 15h30 Report on On-site data processing simulation deliverable slides (Ari Lukkarinen, CSC)

No comments to the DO2 modifications. The SG has already agreed to this and the schedule should be worked out by the E3DDS project.

Comment: This project will officially end on April 30th 2020. There is no continuation foreseen after this date, therefore the benefits of this project must be handed over to someone.

Comments from the Stakeholders present: None.

  • 15h40 - 15h45 Wrap-up, next meeting, propose April 7th 2020 (Tomasz Malkiewicz, NeIC)

Decision: Tuesday, April 28th 2020 Umea. Starting time: 10h00 with coffee from 09h30. End at 16h00.

Retrieved from "https://wiki.neic.no/w/ext/index.php?title=E3DDS_Stakeholder_20191017&oldid=11514"