E3DDS Kick-off meeting

From neicext
Jump to navigation Jump to search

E3DDS Kick-off meeting

Time: Monday 2018/05/07 at 10h30 CEST
Place: Ångströmlaboratory, Uppsala University
Room: ITC 1145 "hus 1" across the road from the Ångströmlaboratory. Building in the background of: https://www.space.irfu.se/workshops/EISCAT-3D_User2013/

Useful links

Projected Attendance (boldface: also attending meeting dinner 2000)

John White, NeIC
Ilkka Virtanen Univ. of Oulu FI (later)
Mikko Orispää, SGO FI (later)
Carl-Fredrik Enell, EISCAT SE
Ingemar Häggström, EISCAT SE
Mattias Wadenstein, Univ Umea SE
Ari Lukkarinen, CSC FI
Roy Dragseth, Univ. Tromso NO
Dan Johan Jonsson, Univ. Tromso NO
Steinar Trædal-Henden, Univ. Tromso NO
Janos Nagy, NSC, SE
Yasunobu Ogawa, NIPR JP
Per-Olov Hammargren, SNIC SE
Craig Heinselman, EISCAT SE
Anders Tjulin, EISCAT SE (flight cancelled)
Tomasz Malkiewicz, NeIC

Agenda

  • Meeting chair: John White
  • 10h30 - 10h45 Welcome (Tomasz Malkiewicz, NeIC) slides
  • 10h45 - 11h00 Introductions (All)
  • 11h00 - 11h30 Overview of the project (John White, NeIC) slides
  • 11h30 - 12h30 Lunch
  • 12h30 - 13h00 Access to prototyping hardware
  • 13h00 - 14h00 Setting up detailed plans for the first couple of deliverables.
    • Deliverable 1: SNIC
    • Deliverable 2: CSC
    • Deliverable 3: UNINETT
  • 15h00 - 15h30 Coffee
  • 15h30 - onwards Hardware for second level beamformer. Some info to be found here

Meeting notes

NeIC introduction

Tomasz Malkiewicz, NeIC

General overview from the NeIC point of view of the project that hopefully runs with the minimum of bureaucracy.

No questions.

Introductions

Three "new" people present:

  • Dan Johan Jonsson, Univ. Tromso NO
  • Steinar Trædal-Henden, Univ. Tromso NO
  • P-O Hammargren: SNIC SE, observer

Overview of the project

John White, NeIC

Questions:

Roy: Do we have a software architect?

Craig, Carl-Fredrik: There is a group that will meet with Jussi to meet in Kiruna in a couple of weeks. One task is to define reqs. for people to hire. Jussi will stay in Kiruna for a few weeks.

Craig: The second beamformer is identical in principle to the first stage beam formers (per FPGA board). We go for software on standard technology for flexibility for the scientists to be able to change how the second beam former is programmed and operated. Currently GPU and FPGA for the second beam former not a primary option. Also price considerations.

Craig: Shows the animations of how the beams would look.

(Get these for "standard" talks!)

Access to hardware for prototyping

Comments from each of the national computing providers on access to hardware for testing/prototyping purposes during this project.

Mattias: We need High Throughput computing nodes with fast interconnect (for delibevables 1 and 2) and storage (for deliverable 3).

Roy: We will use the normal resource allocation at Sigma2. Will need a Norwegian PI (UiT). Not a problem as plenty of candidates.

What are the computing requirements that EISCAT needs looking at?

Ingemar: The 5 MHz bandwidth for first 5 years relaxes demands on the online computing.

Mattias: The beamforming should be straightforward. WAN transfer demanding. No need to buy petabytes of storage for this. Would add data archives to the project, though

Craig: We don't need to combine the data from all sites at one place on day 1 of operations. This will come later (when?).

AL: A question is how to run tasks, use Openstack?

(A bigger question of if the online computing can be virtualized needs to be answered in the project)

Also some comments on general access to computing for users.

Per-Olov: Access to national-owned resources may be tricky for non-Swedish PIs...

Craig: It's important to remember that ALL EISCAT users will need access to computing/data. So people will be needing access from outside the Nordics

Mattias/Roy: Solution is to add some computing to already-existing facility and guarantee EISCAT users access to such...

Plans for Deliverable 1

Mattias Wadenstein, SNIC


  • Need to collect the on-site computing requirements. POSIX compliance on the site storage?
  • Also collect a matrix of the cases for the beam former:
    • Beams per sub-array
    • Number of narrow-angle beams
    • Band-width
    • Filter length(s)

ACTION: Need to set up a meeting with the correct people present to get the "final" requirements on the first to second beam former, ring buffer, second beam former. Mattias will set up doodle. Need Assar, Jussi, Ingemar, Mattias...

Plans for Deliverable 2

Ari Lukkarinen, CSC

  • Virtual machines/containers... how feasible are these for the online computing? This needs to be tested.
  • User analysis... given the priorities stated above... how to help out with this.
  • Still need to ask some questions on how the user analysis will work.
  • Should buy some resources. The Umea computing model is a good model.
  • For the access to hardware we can get the resources.
  • Deployment: We need to assess VMs, containers, bare metal. Other technologies?

Roy: There is a software development part not to underestimate. Is MPI fault-tolerant?

Question: How does the second beamformer run? Does it run all the time or not? i.e. process that reacts to data incoming stream?

Carl-Fredrik: We also have to consider the scheduling of multiple experiments and do the beamforming processes have to be modfied?

Ari will start to sketch scenarios.

Plans for Deliverable 3

Roy Dragseth, UNINETT

  • We have the challenge of the demanding data transport, up to 53 Gb/s if we can test full scale. Not for prolonged periods though.
  • Network topology: Simulation running at Umea, Tromso and Helsinki (Kajaani)
  • Need to study the on-site storage technology (Lustre, GPFS, Ceph,..., ?)
  • Central storage (specific technology, POSIX, ... ?). The central storage is not such a high priority as stated above.
  • Priorities: The on-site storage tech, data-flow test, requirements for central storage wide-area tests not very highly prioritized. (Good news)

Comment: We could include some studies on the transfer to the data archive. Gives some more questions to be asked/answered.

Operations profiling page

(All)

  • The central disk buffer section we can "drop" for now.
  • More priority on:
    • Organization
    • Site storage
    • How EISCAT_3D plans to operate
    • Real time processing
    • Data Centres

ACTION: John to update the page and continue the question gathering exercise. For future meetings.

Questions:

Roy: For procurement and delivery? Delivery to Kiruna then to NO and FI? Or delivery to FI,SE,NO separately

Craig: Delivery is to each country separately

Question: Capacity for processing and rendering? 50 TFLOPS user processing.

Roy: It's easy to apply for resources within Nordics esp Sigma2 (free for NO PIs). Also some comments on general access to computing for users.

Per-Olov: Access to national-owned resources may be tricky for non-Swedish PIs...

Craig: It's important to remember that ALL EISCAT users will need access to computing/data. So people will be needing access from outside the Nordics. EISCAT must not have different policies for scientists in different countries. Otherwise we run into big difficulties.

Mattias/Roy: Solution is to add some computing to already-existing facility and guarantee EISCAT users access to such...

Carl-Fredrik?: We could use of one EISCAT VO for all Eiscat researchers would prove identity, allowing access to whole Eiscat community.

Question: Any restrictions vs citizenship?

Per-Olov: SE/SNIC rules do not allow gathering citizenship info. ITAR rules do not allow citizens of certain countries into machine rooms.

Hardware for second level beamformer

The discussion continues... general review of the previous design.

There will be some numbers to determine given bandwidths and also the number of beam required OUT of the second beamformer as some experiments may require MORE than 100 narrow beams!

We set up a doodle for the future meeting.

Retrieved from "https://wiki.neic.no/w/ext/index.php?title=E3DDS_Kick-off_meeting&oldid=9494"