Glenna2

From neicext
Jump to navigation Jump to search
NeIC Glenna Cloud

Status/phase

The Glenna2 project was the continuation of the Glenna1 effort https://wiki.neic.no/wiki/Glenna

Status: Ended

The Glenna2 project finished in March 2020.

Aim

Glenna 2 aimed to provide added value to the Nordic national cloud and dataintensive computing initiatives by:

  • Supporting national cloud initiatives to sustain affordable IaaS cloud resources through financial support, knowledge exchange and pooling competency on cloud operations.
  • Using such national resources to establish an internationally leading collaboration on data intensive computing in collaboration with user communities.
  • Leveraging the pooled competency to take responsibility for assessing future hybrid cloud technology and communicate that to the national initiatives.
  • Supporting use of resources by pooling national cloud application expert support and create a Nordic support channel for cloud and big data. The mandate is to sustain a coordinated training and dissemination effort, creating training material and providing application level support to cloud users in all countries.

People

Team

Weekly meetings: Glenna2 team meetings

Please see the NeIC official website.

Steering group

Meetings, 3-4 per year: Glenna2 steering group minutes

Please see the NeIC official website.

Reference group

Meetings: Glenna2 reference group minutes

Please see the NeIC official website.

Management

Meetings, Weekly: Glenna2 management meetings

  • Project owner: Michaela Barth
  • Project leader: Dan Still


Results

Workshops

  • Glenna2 Kickoff 2017-06-07 Arlanda Sweden. See agenda and details on the event wiki page
  • UPPMAX CSC Cloud & HPC Workshop 2018-04-27 CSC Espoo Finland. See agenda and details on the event wiki page
  • Glenna2 Workshop and F2F 2019-09-19 Arlanda Sweden. See agenda and details on the event wiki page
  • Glenna2 Final F2F End of Project Workshop 2020-03-10 - 2020-03-11 Uppsala Sweden. See agenda and details on the event wiki page

NeIC All Hands January 2018 Glenna2 Presentations

NeIC All Hands January 2019 Glenna2 Presentations

Newsletters

Other news

Outreach

Note: travels to events and representations are also collected in the internal part of the wiki

Visited events

Name Start date End date Description Particpated Links
Presentation at 3rd SIG CISS Berlin(DFN) 2018-04-12 2018-04-12 K.Happonen: "Rally and Tempest. Experiences with automated testing against your OpenStack deployment" GEANT NREN cloud experts [1]
EOSC-hub Week, Spain 2018-04-16 2018-04-17 Glenna presentation given during EOSC-hub Week EOSC community [2]
Cloud Panel at ECMWF HPC Workshop Reading UK 2018-09-27 2018-09-27 Title: Convergence of HPC and the Cloud HPC Experts - Worldwide Met Office Community [3]
NeIC All Hands Meeting 2019-01-28 2019-01-31 Meeting for all NeIC staff Glenna2 staff [4]
CSC FINLAND CIO Network webinar presentation 2019-05-06 2019-05-06 Dan Still: Cloud presentation for Finnish CIO:s at univeristies - Webinar part of FUCIO network activities Finnish CIO:s [5]
Rahti Presentation 2019-05-15 2019-05-15 Risto Laurikainen: Introduction to the Rahti container cloud - hands on session NeIC 2019 Conference participants [6]
Glenna2 Presentation 2019-05-15 2019-05-15 Dan Still: The Glenna project NeIC 2019 Conference participants [7]
Presentations from the AE meeting @ PDC KTH Sweden 2019-xx-xx 2019-xx-xx Salman Toor: SNIC Science Cloud SSC cloud users [8]
GÉANT Infoshare October 2019 2019-10-23 2019-10-23 Dan Still: GÉANT Community Clouds (includes description of Glenna environment) GÉANT project members [9]
Nordunet Workshop 2019 2019-09-25 2019-09-25 Dan Still: GN4-3 Community Clouds Nordic NREN:s [10]

Documents

Deliverables

  • Aim3: Target 4 - D1: Install and optimize the DL platform for enabling the neural diagnostic workflows (NO).
Collecting input data for this use case proved challenging and focus was shifted towards the Lean AI Stack effort referenced below.

Open Source

Scaleout.jpg


The Lean AI Stack is a open source project aiming to be a complete solution for working on End to End machine learning. From experiments and exploring datasets to large-scale training to end user serving and monitoring of models and their performance in production. In addition to supporting the basic workflow of finding good models it also support extensive customization and adding value through setting up automated machine learning pipelines. As part of Aim-3 Target 4: first Glenna2 co-funded AI “software effort” in collaboration with industrial partner Scalout Systems AB.

https://github.com/leanaiorg/leanaistack

Booklet

Articles

  1. Capuccini M, Toor S, Arvidsson S, and Spjuth O MaRe: Container-Based Parallel Computing with Data Locality GigaScience, Volume 9, Issue 5, May 2020, giaa042, DOI: https://doi.org/10.1093/gigascience/giaa042 (preprint was at arxiv.org/abs/1808.02318)
  3. Capuccini M, Larsson A, Carone M, Novella JA, Sadawi N, Gao J, Toor S, Spjuth O On-demand virtual research environments using microservices PeerJ Computer Science. 5, e232 (2019) DOI: https://doi.org/10.7717/peerj-cs.232
  5. Khoonsari PE, Moreno P, Bergmann S, Burman J, Capuccini M, Carone M, Cascante M,de Atauri P, Foguet C, Gonzalez-Beltran A, Hankemeier T, Haug K, He S, Herman S,Johnson D, Kale N, Larsson A, Neumann S, Peters K, Pireddu L, Rocca-Serra P, Roger P,Rueedi R, Ruttkies C, Sadawi N, Salek RM, Sansone SA, Schober D, Selivanov V,Thévenot EA, van Vliet M, Zanetti G, Steinbeck C, Kultima K, and Spjuth O Interoperable and scalable data analysis with microservices: Applications in Metabolomics Bioinformatics. btz160 (2019). DOI: https://dx.doi.org/10.1093/bioinformatics/btz160
  7. K. Peters, J. Bradbury, S. Bergmann, M. Capuccini, M. Cascante, P. de Atauri, T. M. D. Ebbels,C. Foguet, R. Glen, A. Gonzalez-Beltran, U. L. Gu ̈nther, E. Handakas, T. Hankemeier, K. Haug,S. Her- man, P. Holub, M. Izzo, D. Jacob, D. Johnson, F. Jourdan, N. Kale, I. Karaman,B. Khalili, P. E. Khon- sari, K. Kultima, S. Lampa, A. Larsson, C. Ludwig, P. Moreno,S. Neumann, J. A. Novella, C. O’Donovan, J. T. M. Pearce, A. Peluso, M. E. Piras, L. Pireddu,M. A. C. Reed, P. Rocca-Serra, P. Roger, A. Rosato, R. Rueedi, C. Ruttkies, N. Sadawi,R. M. Salek, S.-A. Sansone, V. Selivanov, O. Spjuth, D. Schober, E. A. Thevenot, M. Tomasoni,M. van Rijswijk, M. van Vliet, M. R. Viant, R. J. M. Weber, G. Zanetti, and C. Steinbeck PhenoMeNal: Processing and analysis of Metabolomics data in the Cloud Gigascience. 8, 2, giy149. (2018). DOI: https://doi.org/10.1093/gigascience/giy149
  9. Spjuth, O.; Capuccini, M.; Carone, M.; Larsson, A.; Schaal, W.; Novella, J.A.; Stein, O.; Ekmefjord, M.; Di Tommaso, P.; Floden, E.; Notredame, C.; Moreno, P.; Emami Khoonsari, P.; Herman, S.; Kultima, K.; Lampa, S. Approaches for Containerized Scientific Workflows in Cloud Environments with Applications in Life Science. Preprints (2020). DOI: 10.20944/preprints202001.0378.v1 https://www.preprints.org/manuscript/202001.0378/v1

Links

References

  • "Apache Spark Streaming, Kafka and HarmonicIO: A Performance Benchmark and Architecture Comparison for Enterprise and Scientific Computing": on HarmonicIO streaming engine developed within in a research project. Also contains practical comparisons to Spark, Kafka on SSC resources.

Attachments

Retrieved from "https://wiki.neic.no/w/ext/index.php?title=Glenna2&oldid=12749"