EGI CF14 Nordic accounting workshop session

From neicext
Jump to navigation Jump to search

This page contains information on the Nordic accounting workshop session organised by NeIC session during the 2014 EGI Community Forum 19-23 May.

Time: 16:00-17:30 on Wednesday 21th
Place: Great Festive Hall, EGI CF 2014, Helsinki, Finland.
Audience: Open to all registered at the EGI CF.
Contact person: Michaela Barth.


Description

Consolidated and consistent accounting within the Nordic computing centres is considered as one of the top-priority topics within Nordic e-infrastructure. Besides the continued SGAS roadmap discussion the main purpose of this workshop is to increase the knowledge exchange on accounting between the Nordic countries and to serve as a framework for possible collaboration partners to lift further accounting projects to a Nordic level.


Agenda

This session serves as a follow-up of the Nordic accounting workshop 2013.

Session Chair: Michaela Barth, NeIC Generic Area Coordinator

  • (CANCELED) 16:00 - 16:15 Accounting on the NHPC Pilot Gardar, Anil Thapa (Technical coordinator, NHPC) (CANCELED):
Simple approach that NHPC adopted for accounting with participating countries will be updated, which will then followed by lesson learned from the practice.
  • 16:00 - 16.15 SGAS today and tomorrow, Magnus Jonsson (NeIC SGAS Developer, HPC2N), slides:
Giving an overview of the current status of SGAS after six months in NeIC care. What has been done and what to do next.
  • 16:15 - 16:30 Update on SAMS and SUPR, Björn Torkelsson (SAMS, HPC2N) and Daniel Nilsson (SUPR, C3SE), slides:
We will give a short status update on the SAMS (SNIC Accounting And Metrics System) and SUPR (Snic User and Project Repository) projects in SNIC. The emphasis will be on how the collected statistics (i.e SAMS) are used in SUPR and how it is presented for the projects within SUPR. We will also give a brief update on what's next for the SAMS and SUPR projects.
  • 16:30 - 16:45 TTA services presentation, Tuija Raaska (Development Manager, Metadata services, CSC):
The Finnish Ministry of Education and Culture provides services for actors within the Finnish research system (ATT Open data and research initiative). Some of the services CSC has built as part of this are already available, work on some others is still in progress. One of the currently available services is IDA, the centralised research data storage service. Since last year we also have had the AVAA publishing platform available. Recent news is that KATA metadata catalogue service is coming to join the services within weeks. We also maintain a very popular research data management guide. Related websites: tdata.fi, avointiede.fi, avaa.tdata.fi, kata.csc.fi
  • 16:45 - 17:25 SGAS Roadmap discussion, Magnus Jonsson (NeIC SGAS Developer, HPC2N):
Confirming that the releases given with Milestone 1 and Milestone 2 correspond to prior roadmap discussion and define the extent of Milestone 3 and 4
  • 17:25 - 17:30 Session Summary, Michaela Barth (NeIC, PDC):
Summarizing the discussions, setting a date for the next Nordic accounting workshop in autumn and closing the session.


Minutes

  • SGAS has the possibility to store multi-core jobs, not only for Grid-jobs, but for all kind of jobs.


  • Graphical tool wishes:
    • NT-1 doesn't have a flexible graphical interface. "How many simulation jobs are we running on this site" should be represented by a graphical interface. The graphic tools in core SGAS are not very good, but this should be possible: The Graphs shown in the SAMS and SUPR presentation are based on aggregated data that is transported once a day. For usage per account we have real-time data as well. The data is in the database, this is just an interface question. SUPR has a custom query interface towards SGAS.
    • For standard reports (per month, per year, per site.) we could use tools like High-chart, for more advanced, flexible query options there are systems for statistics gathering which can do standard reports but also more advanced querying, to run a query that is not the usual one.
    • Oxana stated that this would also be interesting for the Tier-1 Tier-2 difference: "Our users have better dashboards than we. We have to trust the users no matter what they say. So actually it is an important tool."
    • Consideration on what should be kept within SGAS, or what should be done within SAMS instead. Possibly better to keep core SGAS slim.


  • Connection to APEL accounting system:
    • NT-1 and Tier-2s have a script that takes data from SGAS and sends it to APEL. Stuart remarked that there are small changes in the scheme-out for the format. Erik will sync this with him.
    • Storage accounting: The newest implementation in the upcoming release should be fully compatible to StAR 1.2. The current implementation in SGAS was based on an early draft (htj was part of the team that wrote the basic star schema).
    • SGAS collects the data and reports when it is done. It is not suitable to use the accounting directly from the sites.


  • SGAS Road-map discussion:
    • Outcome see below
    • Decision to put the road-map also on GitHub.


  • Date for the next Nordic accounting workshop:
    • Lunch2 Lunch on 16th-17th of October in Uppsala (back2back with the Application Expert meeting there the days before)


SGAS Road-map

Common

  • user group meetings together with EGI community forum (19-23 May)

(ongoing)

Bugs

  • (M2) CPU duration handling wrong way from usage records in Jura

Improvements

  • (M3) get a plugin infrastructure so add-ons can be added
    • M1 had already included a contributor directory!
  • (M4)Extend the reporting views -> if it takes time, it can take time, more important to get this done the right way
    • separate module(s)?
    • removing existing views???
  • BART
    • (M2) Client side verification (Add verification step into bart-reporter & storage-reporter)
      • bad files
      • empty files
      • bad records within files
      • Missing hostname?
      • tool for verify star records.
    • Change BART archive directory
    • Lots of files problem (BART uses one directory per usage record)
    • Support of LRMS? Slurm, Moab, Gold
      • support for SLURM and Torque already included, but problems with newer versions of LRMSs
      • Not supported -> should be community provided plugin.
  • (M3) make idtimestamp default!, might be already done
  • (M4) Cloud accounting (UR-2)
    • (M3) what can the cloud stacks use? --> check
  • (M3) Documentation review
    • remove wrong documentation!

Extension

  • (M3 -> no rather SAMS contribution) Migrate SAMS custom query interface into mainstream SGAS
  • Migrate NeIC APEL reporting into mainstream SGAS
    • SSM2?
    • contribution might be best choice, rather specific and not of generic use
  • (M3) Command for remove records for a cluster in a safe way
  • Host scale factors.
    • Put WLCG views into separate module??
    • (M3)-> remove it from the core and make plugin instead.

Links

Attachments

Retrieved from "https://wiki.neic.no/w/ext/index.php?title=EGI_CF14_Nordic_accounting_workshop_session&oldid=1640"