Glenna/SG-Meeting-2016-02-29

From neicext
Jump to navigation Jump to search

Glenna steering group meeting

Time: Monday 2016-02-29 at 09:30-15:00 CET
Place: Arlanda, SkyCity

Approved: 2016-03-12


Invitations:

  • Martin Bech, representing DeIC, DK
  • Andreas Hellander, representing SNIC, SE
  • Hans A. Eide, representing UNINETT Sigma, NO
  • Sæþór L. Jónsson, representing CSUI, IS
  • Jura Tarus, representing CSC, FI
  • Dan Still Project Manager representing NeIC
  • Dejan Vitlacil, NeIC Generic Area Tecnical Consultant, representing NeIC
  • Michaela Barth, NeIC Generic Area Coordinator, representing NeIC, Chair


Links

Agenda

Glenna SG 16-01. Welcome and Presence.
Glenna SG 16-02. Approval of agenda and minutes from last meeting
Glenna SG 16-03. Project status report
Glenna SG 16-04. Project results of DP5/6: Demonstration of services and selected pilots.

Lunch

Glenna SG 16-05. Decision on continuation of project
Glenna SG 16-06. New Version of Project plan
Glenna SG 16-07. Marketing and Target audience for newsletter
Glenna SG 16-08. Information on Nordic Cloud Collaboration Planning Group
Glenna SG 16-09. Future strategy
Glenna SG 16-10. AOB
Glenna SG 16-11. Next meeting

Minutes

Glenna SG 16-01. Welcome and Presence

(5 min)

Decision:

Everyone present

Glenna SG 16-02. Approval of agenda and minutes from last meeting

(5 min)

Decision:

Agenda and minutes from last meeting approved

Glenna SG 16-03. Project status report

(20 min) Project_Status

PM_Allocation

Status overview of Glenna goals: Status and progress towards the project goals in attached pdf.

Green=good progress Red=not good progress

Discussion:

The opening of the UiO Lifeportal to Kalmar2 has been promoted by NeIC personnel at the NIASC annual meeting in Copenhagen mid-January.

Sweden has provided approx 16PM:s up until now. SNIC will send bills in the future.

Meteo MUSC software just set-up for Sweden so far, but the idea is to run it on the Norwegians met.no's own OpenStack.

Regrettably it seems that the project will have quite a bit of work to enable access to the services throughout all of the sites in Kalmar2. There are challenges with Kalmar2 enabling because of varying practices/policies between sites. We can wish this could be worked and improved on in the future.

On a side remark: the knowledge sharing within Glenna has led to internal discussion on Seafile vs ownCloud solution at CSC. Members of the CSC long term storage team discussed above issue with DeIC personnel in VC.

Decision:

Status approved

Glenna SG 16-04. Project results of DP5/6: Demonstration of services and selected pilots

(90 min)

Demonstration of the key Glenna services: DeIC data cloud, UiO Lifeportal and CSC's Pouta Blueprints. Olli Tourunen presented Pouta Blueprints (over video), Gurvinder Sing presented Kubernets (over video) and Salman Toor presented the SaaaS use case (in person). Andreas Hellander demonstrated the usage of the MOLNS network to submit to Amazon and as well to Swedish cloud resources as to Finnish cPouta within Stochastic Simulation Service (StochSS). Dan presented an overview of the results as well as the funding scheme and the template for the Service Level Agreement (SLA).

The METEO use case instructions for installing and running a real life multi-million line NWP code in a Nordic Cloud that were attached to the agenda, were acknowledged but not discussed in further detail.

Glenna_DP56_Results.pdf

METEO_Cloud_Installation.pdf

  • Dan: Glenna service and funding scheme:

Funding Scheme.pdf

SLA example.pdf

Discussion:


Olli Tourunen: Pouta Blueprints

The service is not reachable via Kalmar2 yet. A distributed filesystem could be used to make the iPython notebooks (a special case of Jupyter) persistent, currently the data has to be exported to the own computer. The choice of persistent storage is not unproblematic since it has to be secure and reliable. The first external user bug reports came in a week ago. CSC is hiring one person to replace the person who left.

Gurvinder Sing/Olav Kvittem: Kubernets

Kubernets was used when detecting gravitational waves. Kubernets is using an elastic infrastructure. This new design focuses on high availability aspects. The Hybrid Cloud model sets the user free to use whatever type of cloud he wants: bare metal, OpenStack, Google or Amazon.

Kubernets is also used in a European meteorological institution which runs on a cloud in Switzerland. Gurvinder asks to Dan to put him into contact.

The SNIC Cloud works on providing kubernets as a service within it.


Salman Toor: SaaaS

Each node in the initial LoI application has contributed one scientific application:

  • Compact Muon Solenoid (CMS) from HIP

The CERN experiment based CMS have tested multi-cloud environment based on Pouta and University of Helsinki's own cloud. The core components are: ARC middle ware, CVMFS and the HIP protocol. The goal was that the users should be oblivious whether their jobs land on cloud or grid resources.


  • Long time Scale Simulations of Solids (EON) from UoI

EON is concentrating on diffusions of atoms and chemical reactions. It was written in C++ and python and designed for HPC resources. Now EON is using Pouta resources.


  • Stochastic Simulation Service (StochSS) from UU

Andreas Hellander is the PI of StochSS. StochSS is using the MOLNS framework to submit to either Amazon or OpenStack (see below).


Andreas Hellander: StochSS in cPouta (SaaaS Demonstration)

https://86.50.170.19:1443/login Andreas demonstrated to run on clouds both in Sweden and in Finland. This can be done for any Amazon or OpenStack cloud. Federated access is important when all are using default OpenStack installs. Storage access is currently broken, but everything is already abstracted in the software by setting flags.

Using ObjectStore directly could be a possibility, since it is key in sharing large datasets and making them available for analysis.

This is also a demonstration of "interactive parallel computing": Using the same VM, common visualisation, sharing parts of code as load models and sharing common software space makes collaboration much easier.


Dan Still: SLA and Funding Scheme

Lifeportal support: so far everyone would get (basic) support regardless of where situated. There is a difference between basic support and advanced support. Advanced support should be tailored. If hosted at CSC (where no difference is made on the level of support) a distinction would be made whether the project is funded by the government or not. The current proposed SLA is missing such a differentiation on the kind of quality of the support.

NeIC is not its own legal entity, should be NordForsk instead.

Towards a future production service: To bundle IaaS services is very difficult. It is also wrong to start common IaaS. There is no intention whatsoever to have a single administrative entity to manage the users, the management structure should reflect the distributed structure of the services. Still there might be one entity needed to keep track of who is using which resource, and also to keep track of who is supposed to use the resources. Project requests will have to be handled by some central entity. Can't just log in to any IaaS resource. There needs to be some entity that keeps track of how much people are using i.e. some type of resource allocation. Easy access is one aspect but we also need a second layer where we can manage the usage of resources(who&what). Sweden uses the OpenStack regions concept. Virtual currency (tokens) in Sweden, in theory tokens could be offered to a future "NeIC Glenna" cloud. There could be a small default quota but access to IaaS should be scrutinized before letting in users. In Norway Sigma2 has allocated (bought) a million CPU hours towards the Lifeportal. Everyone gets immediately a certain amount of quota (100 hours) to get started with the system and test its capabilities. The users have to apply for more.

In Denmark the plan is to stand on two legs: commercial and academic IaaS. In Uppsala there is the SNICsense project for handling sensitive data. Would probably be an expensive solution to go for a completely commercial solution public cloud for Nordic users.

The plan in Uppsala is to spend 2-2.5FTE on maintaining the OpenStack cloud SMOG. Accurate pricing is needed in order to exchange services. SNICcloud will likely not have an explicit SLA. Instead operate on best effort.

Lifeportal: Not earning money, we should be able to bill the real cost. The real cost is known. There is a contract with Sigma that says how much it costs to operate Abel which includes the Lifeportal. Pricing would be linked to CPU hours spent and whole cost of ownership.

CSC: Big allocations need an application, but we can't kill the flexibility of cloud usage. This flexible usage still needs to be possible.


The projects central message is that there needs to be an governing entity. The Board might appoint some entity. This entity could report to the Provider Forum.

Billing/accounting might be easier to approach through the proposed new NeIC HPC sharing initiative, and that the principles/framework developed there might provide a anchor-point to extend to cloud services.

Question to all: How interested would the partners really be in offering services to other Nordic centers and users and in return receive services?

  • SNIC: It was hard to get the SNIC cloud project approved, it is still a project, not base infrastructure. Andreas thinks Sweden is positive to resource sharing and we would likely support it. But don't try to invent another Grid!
  • DeIC: no IaaS in Denmark, we would support it
  • Norway: we would support it, if usage picks up that would be a luxury problem
  • CSC: not much capacity to give out: already now: too much usage on our computing resources, right now we don't need Glenna for selling our services. But there is of course the life cycle of resources to be taken into account.
  • Iceland: we would support it. Iceland profited from the knowledge gained when hosting the NHPC pilot and is ready to again host any equipment for you.

Glenna SG 16-05. Decision on continuation of project

(10 min)

DP 5/6 decision for SG: Go/No-Go?

Decision:

Many of the problems in the past have been solved. The SG concluded that it would be a mistake to stop now, especially as NeIC already starts to prepare for a possible continuation. We could declare the project a success already now just for the knowledge exchange and built-up that is so mostly needed.

Go i.e. approved, project continues.

Glenna SG 16-06. New Version of Projectplan

(20 min)

The Project plan has been updated to reflect the current status of the project. Please also consult the attached appendices.

Glenna_project_plan_v1.1.pdf

Appendix A Gantt Chart: .pdf

Appendix B Budget: .pdf

Appendix C Glenna PM Allocations per Country:.pdf


Discussion:

The striking of T2.4 is a pity, but an obvious decision. T4.2 should be further updated.

Decision:

Project plan updates approved

Glenna SG 16-07. Marketing and Target audience for newsletter

(30 mins)

SG opinion and comments on marketing e.g. newsletter and new Glenna Portal. The portal is work in progress and the long term permanent location for the portal is not decided.

https://wiki.neic.no/w/ext/img_auth.php/f/fc/Glenna_Newsletter_January_2016_PDF.pdf

https://wiki.neic.no/w/ext/img_auth.php/8/8e/Glenna_Newsletter%2C_February_2016.pdf

https://wiki.neic.no/wiki/Glenna_Portal


Discussion: email lists: SNIC office do not have a email list. They use the SUPR list. The marketing for the SNIC Cloud has just started. There could be a press release just for the services that are working. Then the national HPC centers could pick that up through their own channels. Ask the centers to market through their own channels. PDC has pdc-announce list.

Suggestion: Training is the best marketing, The key for Marketing is getting users in, so we can learn from the experiences.

Webinars: Video will scale with the participant number.

Workshop on data analytics: We could replicate popular workshops.

Glenna was invited to Bergen conference in early May

Dan will go to ECMWF to give a talk on Glenna in Reading in the UK

Salman could mention Glenna when talking about OpenStack.

A few general slides about Glnna would be needed for members giving talks is needed. Dan demonstrated the Glenna portal. The NT1 webpage is in the WIKI.


Decision: A few general slide about Glenna needed for members giving talks is needed.

Glenna SG 16-08. Information on Nordic Cloud Collaboration Planning Group

(30 mins) This agenda item concerns the potential follow-up project to Glenna in the context of the Nordic eScience Action Plan 2.0. Chair presents


Discussion:

PO presented draft for cloud collaboration planning group ToR which will be discussed at the next NeIC Board meeting 9.3. and follows the eScience action plan 2.0. Intention is to form a planning group which will continue with planning a follow up project on cloud services. The experiences from Glenna should go on to this working group which will decide where to set focus.

The SG commented positively on the amount of FTE and the tight timeline which is important to keep momentum and not to loose the cloud competencies built-up within Glenna. So staffing should be easier next time.

Andreas: if we do some cloud collaboration in the future the infrastructure providers should make sure they anchor their commitment in their own organizations. (Everybody agreed)

  • 2-3 physical meetings and produce project directive.
  • Participants should have substantial time allocations
  • Committing staff before project starts is important
  • Collect intentions from centers how future project would fit into national strategies
  • Differentiation between commercial providers and future "Glenna"?

There will likely be a combination of public and private cloud. More thoughts should go in the long-time funding: Who should be billed? Should the services offered appear free to the end-user?

Recommendation to the Board: A continued cloud project should be matched by national efforts in the NeIP's roadmaps.

Glenna SG 16-09. Future strategy

(30 mins) This agenda item aims at refining the objectives of the project.

Glenna_Strategy_for_Future.pdf

Discussion: Agenda item not discussed

Decision: N/A

Glenna SG 16-10. AOB

(15 mins)

  • NeIC is trying to establish (not enforcing) a new policy on openness and approachability, NeIC strives to make basic contact information available for all people that play a part in NeIC activities also project steering group members. Basic contact information generally means:
    • Name
    • Home institution
    • Role
    • Email address
    • Photo (optional)
  • Michaela asks for allowance from each of the SG members to publish this basic contact information.

Decision: No contest


. Managers will now have to report Key Performance Indicators (KPI:s) to the NeIC Board also including accurate project time expenditure.

  • Question: should time reporting be introduced in the Glenna project?

Discussion: In all EU projects quarterly reports are done. The PM asks and the sites respond. 90% is not in all cases possible because of dependencies. There might not be enough work to do, because of delays. work can fluctuate between 80 and 110% Worry is that time reporting becomes so important that participants will report even though no work has been done. 90% of personnel resources is not a good KPI. The KPI:s will be discussed during the next Board meeting. SNIC: risk of micro management. Interest in NeIC project participation may diminish. Will lead to more reporting but not to more actual work. Time reporting creates too much overhead.

Other NeIC projects are happy with counting assigned commitment per National Infrastructure. EU project time reporting was mainly introduced to avoid more than 100% overcommittment.

Decision: Send message to NeIC Board: Glenna project is not in favor of the 90% rule. If NeIC Board decides the KPI stays at 90%, quarterly time reporting will be mandatory (one or two lines of executed work + total amount of hours per month/quarter)


  • Question: Should we merge cPouta and SNIC Cloud?

Glenna SG 16-11. Next meeting

(5 min)

Decision: Wednesday 25.5.2016 in Oslo. (Radisson)

Jura Tarus will attend over video. Could invite Pål Petterson

Attachments

Retrieved from "https://wiki.neic.no/w/ext/index.php?title=Glenna/SG-Meeting-2016-02-29&oldid=5693"