Dellingr-SG-meeting-2018-05-29

Dellingr Steering Group Face-to-Face meeting

Time: 2018-05-29 14:00 - 16:00 CEST

Place: CSC, Espoo, Finalnd. Video participation: https://cscfi.zoom.us/j/9917669499

Live minutes: https://docs.google.com/document/d/12ed5WmzaU1US2hmfRwHDwJqa1t2Qs2KhRAbNNwUhn5E/edit?usp=sharing

Invitations:

• Kurt Gammelgaard Nielsen, DK
• Per-Olov Hammargren, SE
• Heta Koski, FI
• Ebba Þóra Hvannberg, IS
• Ivar Koppel, EE
• Jørn Amundsen, NO (Observer)
• Tomasz Malkiewicz, NeIC, Project owner
• John White, NeIC, Project manager
• Gudmund Høst, NeIC, Guest
• Rob Pennington, NeIC, Guest

Agenda :

18-3-4 Partners' contributions to the project (for decision) 20'

Revised effort for the project: this link. The effort indicated is in person-months. The Gantt chart tries to show the logical progression and relation of the deliverables to each other.

18-3-11 AOB 5'

Minutes :

18-3-1 Presence, approval of the agenda (for decision) 5'

(Tomasz)

• Presence:
• Kurt Gammelgaard Nielsen, DK
• Per-Olov Hammargren, SE
• Heta Koski, FI
• Ebba Þóra Hvannberg, IS
• Ivar Koppel, EE
• Jørn Amundsen, NO, Observer (remote)
• Tomasz Malkiewicz, NeIC, Project owner
• John White, NeIC, Project manager
• Gudmund Høst, NeIC, Guest
• Rob Pennington, NeIC, Guest (remote)

• Decision: Agenda approved.

18-3-2 Project status (for information) 15'

(John (PM) and Tomasz (PO))

• The project update from PM
  • Personnel status
    • All members (EE,IS,FI,SE, also NO in meetings!) participating except DK.
      • Still need some input to DO1 from DK.
      • Also need for DO3.
      • Not needed for DO2 (I think).
      • Harder to test against WAYF. (IS help?)
      • Cannot share on Abacus any more.
  • Update on progress towards delivery objects
    • DO1 for approval (see below)
    • Work for DO3 started. The information has been gathered.
    • Will follow the same model as DO1. Sections assigned. Need overall skeleton.
    • DO2 also started. We collect the actual usage in the first pilot.
  • Highlights: From DO1 discussions.
    • Many sharing models discussed.
    • “Model 4” chosen for the model to exchange the resources
    • Benchmarking discussed. Not really useful for users. Mainly for HPC operators to tune their systems.
    • We choose a more user-friendly set of benchmarks from PRACE
    • Use the Juelich Benchmarking environment for repeatability
    • Benchmarks in NeIC github
    • https://share.neic.no established… We investigate compatibility with established systems (e.g. MAS Dataporten)
    • Try to solve the “GPU problem” (exchange rates and understand what it really means to operate GPUs)
  • Project group proposes a second pilot.
    • Gathering information on the objectives of second pilot to be aligned with Dellingr project objectives.
      • How would it differ from the first pilot?
        • Targeting National Providers, e.g., connecting Waldur system to the National providers resources
        • Test how far can we go with this system
        • Running in the same time frame as the previous pilot
        • Hardware characteristics to be exposed
        • Diverse cloud infrastructure
    • Considering:Time frame, resource request, framework, extra data to gather, prototype user agreement, AAI
  • Contribution to DI4R conference
    • Framework prototype demo?
• The project update from PO will include
  • Update on Nordic Open Science Cloud (NOSC) proposal status
  • Collaboration with other projects
    • Meeting with NLPL
      • Not possible to test Dellingr setup on Abel (NO), NO not participating in DEllingr
    • Positive feedback from Tryggve2 on the pilot
  • Collaboration agreement signing progress
    • Everyone except DK has signed the Collaboration Agreement
  • Issues
    • Manpower DK
      • DK Jens will re-join the Dellingr weekly meetings…
      • Proposal: DK participates in Dellingr project with 4 PM total contribution, future ramp-up possible, e.g., Waldur work.
      • Decision: 4 PM total contribution from DK with possible ramp-up approved.
    • NO withdrawal from Dellingr 20.11.2017
      • NO NLPL involvement is to continue as is
      • Access to Abel for Dellingr for testing NLPL solutions
        • Fine for NO. Provide info for import to Abel? Connection technically to Abel? Export SGAS records, contact Andreas Bach <andreas.bach@uninett.no>
      • Continuous resource sharing a problem?
      • SG would like to see the second pilot proposal.

18-3-3 NeIC budget for 2018 and 2019 (for information) 10'

(Gudmund)

• Tight budget situation due to project cycles
• Project proposals in the pipeline for possible startup

18-3-4 Partners' contributions to the project (for decision) 20'

(John, Tomasz)

• Revised effort for the project this link. The effort indicated is in person-months. The Gantt chart tries to show the logical progression and relation of the deliverables to each other.
• DK contribution is equivalent to the current possibility.

18-3-5 Continuation, change or interruption of the Dellingr project (for decision) 15'

(Tomasz)

• Steering group endorses the project.

Decision: Continue the project as is.

18-3-6 Approval of DO1 Resource Exchange Models (for decision) 15'

(John)

https://wiki.neic.no/w/int/img_auth.php/2/2a/Dellingr-DO1-v1.2.pdf

• Add to Summary of DO1 that what is stated there: “Rather than relying on CPU or monetary-based exchange rates, an exchange rate based on resource popularity is proposed.” does not apply to the proposed second pilot nor to the future continuous exchange of resources.
• New solution to be found. First step might be treating cpu hours equally regardless of the cpu model, i.e., Haswell and Sandy Bridge treated equally, having different rate for GPUs, adding storage to the exchange, perhaps also memory usage as in CSC accounting. User support to be possibly added at later stages.

18-3-7 DO7 Establishment of high level information exchange mechanisms - outcomes of high-level planning and coordination meeting (for discussion) 15'

(John and Tomasz)

• Forum established, first meeting held on 29.05.2018.

18-3-8 Approval of adding to the Project Plan deliverable DO10 "Support in enabling NT1 resource exchange (for decision) 15'

• Project Plan v 2.8: https://docs.google.com/document/d/1fVDB_imL8b1s8TRcXj0eYspP-GKd2_Axttf8CBR-lh4/edit
  • DO10:
    • Delivery object 10: Support in enabling NT1 resource exchange
    • Content: Enabling resource exchange for NT1 collaboration. The models developed in the Dellingr project for compensating or balancing resource usage will be applied to the NT1 case. These models can range from summing simple core times and data storage size to sums weighted by the specifications or costs of the shared computing and storage capacity. As an example, computing providers in NT1 that have a possibility for sharing an expanded tape library can provide that in exchange for, e.g. providing less computing. The work will include the study of necessary tools for enabling the exchange and analysis to ensure that the developed models are in-line with the strategies of the countries involved.
    • Recipient: NT1, the national providers and NeIC.
    • Delivery process: Delivery is a document describing the proposed metrics for resource exchanges within NT1, resource exchange mechanism and exchange rates. A group consisting of NT1 experts, NLCG committee chair, NT1 steering group, coordinated by the Dellingr Project Manager working on the metrics and coordinating high-level discussions with the countries involved.
    • Delivery date: November 2018

• Proposal: add it to high-level track
• Decision: Approved.

18-3-9 Status assessment for board (for decision) 5'

• Project on track. NO not participating in the project is a challenge, as the goal is to design and implement resource sharing across the Nordics (all 5 Nordic countries) and Estonia.

18-3-10 Next meetings (for decision) 5'

• Decision: 25.09.2018, 9:30 CEST

18-3-11 AOB 5'

• Waldur meeting between ETAIS, CSC
• Also Waldur discussions between ETAIS and DK.