GFI&SKD Storage: Difference between revisions

From gfi
(Created page with "Coming along with the computing system '''cyclone.hpc.uib.no''' there are also three storage units, each with size of 120TB, and all mounted within the same shared lustre file...")
 
No edit summary
Line 1: Line 1:
Coming along with the computing system '''cyclone.hpc.uib.no''' there are also three storage units, each with size of 120TB, and all mounted within the same shared lustre filesystem. Two of the storage units are funded by GFI, the third is funded by SKD.
Coming along with the computing system '''cyclone.hpc.uib.no''' there are also three raid based storage units, each with size of 120TB, and all mounted within the same shared lustre filesystem. Two of the storage units are funded by GFI, the third is funded by SKD.


The storage areas are available from these main folder paths on cyclone (local access) and any other GFI&SKD linux-client (network acces via nfs):
The storage areas are available from these main folder paths on cyclone (local access) and any other GFI&SKD linux-client (network acces via nfs):
Line 15: Line 15:
[https://it.uib.no/en/Mac_OS_X:_Connecting_to_your_network_share * Mac OS X: Connecting to your network share]
[https://it.uib.no/en/Mac_OS_X:_Connecting_to_your_network_share * Mac OS X: Connecting to your network share]


The storage is organised by priority on larger shared folders for modelling data, in group
The storage is organised by priority on larger shared folders for modelling data, in group and projects folders with and without need for backup and folders for individual use with or without backup. Because backup of large quantities are connected with considerable
and projects folders with and without need for backup and folders for individual use with
annual cost, users and groups will need to consider which datas need a daily backup and which are reproducible and have a reasonably security within a modern raid storage system without backup.
or without backup. Because backup of large quantities are connected with considerable
annual cost, users and groups will need to consider which data need a running daily update
and which are reproducible and have a reasonably security within a modern raid storagesystem.


Most folders assigen for group access have general read-write access on group level. Owners of
Most folders assigned for group access have general read-write access on group level. Owners of each folder and file may still have set exclusive individual access on these folders and files. If other members of the assigned group find this inconvenient, please contact the formal owner of the file or folder.
each folder and file may still have set exclusive individual access on these folders and files.
If other members of the asssigned group find this inconvenient, please contact the formal owner
of hte file or folder.


Individual quotas (inside /Data/gfi/users and /Data/skd/users) are set by default to 200GB and adjusted by request on individual needs - normally < 1TB. Larger amount should be considered for their own project space. Project and group quotas are managed by configured maximal limits of main project/group folders.
Individual quotas (inside /Data/gfi/users and /Data/skd/users) are set by default to 200GB and adjusted by request on individual needs - normally < 1TB, but still in respect for other users needs and limited by the max. size of the parent folder. Larger amount should be considered for their own project space. Project and group quotas are managed by configured maximal limits of main project/group folders.
 
Access on top level folders are often set read-only to keep folder conventions & structure strict and tidy. When in need for storage space and/or adjustments and customisation, please submit issue to:
 
[https://bs.uib.no/?module=issues&action=new&gid=32 * GFI/BCCR group of UiB UserSupport].


Access on top level folders are often set read-only to keep folder conventions structure strict and tidy. When in need for storage space and/or adjustments and customisation, please submit issue to the
[https://bs.uib.no/?module=issues&action=new&gid=32 GFI/BCCR group of UiB UserSupport].


Sizes, quotas, purpose etc. for the variety of folders inside GFI&SKD storage are given below.
Sizes, quotas, purpose etc. for the variety of folders inside GFI&SKD storage are given below.
Line 65: Line 61:
|-
|-
|work
|work
|shared short time(30days) Symbolic link -> /Data/skd/work
|shared short time (30days) Symbolic link -> /Data/skd/work
|33 TB
|33.75 TB
|GFI & SKD
|GFI & SKD
|
|
Line 135: Line 131:
|-
|-
|}
|}
<\br><\br>
 
 
{| class="wikitable"
{| class="wikitable"
!colspan="6"|GFI&SKD Storage folders inside /Data/skd/
!colspan="6"|GFI&SKD Storage folders inside /Data/skd/
Line 147: Line 144:
|-
|-
|share
|share
|Symbolic link ->/Data/gfi/share/
|Symbolic link ->/Data/gfi/share/ - shared model data
|
|
|GFI & SKD
|GFI & SKD
Line 154: Line 151:
|-
|-
|skd-share
|skd-share
|/Data/gfi/share/* -> /Data/skd/skd-share
|Several subfolders /Data/gfi/share/* -> /Data/skd/skd-share/.
|51 TB
|51.25 TB
|GFI & SKD
|GFI & SKD
|
|
Line 161: Line 158:
|-
|-
|scratch
|scratch
|individual and group
|individual and group share
|20 TB
|20 TB
|SKD
|SKD
Line 168: Line 165:
|-
|-
|work
|work
|shared short time(30days)
|shared short time (30days)
|33 TB
|33.75 TB
|GFI & SKD
|GFI & SKD
|
|

Revision as of 10:26, 8 March 2019

Coming along with the computing system cyclone.hpc.uib.no there are also three raid based storage units, each with size of 120TB, and all mounted within the same shared lustre filesystem. Two of the storage units are funded by GFI, the third is funded by SKD.

The storage areas are available from these main folder paths on cyclone (local access) and any other GFI&SKD linux-client (network acces via nfs): 

- /Data/gfi  and /Data/skd

For GFI&SKD windows-clients and mac-clients inside UiB-network, the storage areas are in general available from the following network shares and you can also mount subfolders of these: 

- \\leo.hpc.uib.no\gfi and \\leo.hpc.uib.no\skd

Howto on connecting from Windows/Mac is published from UiB ITA (just change string for correct server and path):

* Win7/Win10: How to connect to a network share

* Mac OS X: Connecting to your network share

The storage is organised by priority on larger shared folders for modelling data, in group and projects folders with and without need for backup and folders for individual use with or without backup. Because backup of large quantities are connected with considerable annual cost, users and groups will need to consider which datas need a daily backup and which are reproducible and have a reasonably security within a modern raid storage system without backup.

Most folders assigned for group access have general read-write access on group level. Owners of each folder and file may still have set exclusive individual access on these folders and files. If other members of the assigned group find this inconvenient, please contact the formal owner of the file or folder.

Individual quotas (inside /Data/gfi/users and /Data/skd/users) are set by default to 200GB and adjusted by request on individual needs - normally < 1TB, but still in respect for other users needs and limited by the max. size of the parent folder. Larger amount should be considered for their own project space. Project and group quotas are managed by configured maximal limits of main project/group folders.

Access on top level folders are often set read-only to keep folder conventions & structure strict and tidy. When in need for storage space and/or adjustments and customisation, please submit issue to:

* GFI/BCCR group of UiB UserSupport.


Sizes, quotas, purpose etc. for the variety of folders inside GFI&SKD storage are given below.

GFI&SKD Storage folders inside /Data/gfi/
Subfolder Purpose Size limit Group assign User quota Backup
share shared model data 20TB GFI & SKD
share/era5 shared era5 data 130 TB GFI & SKD
scratch individual and group 25 TB GFI
work shared short time (30days) Symbolic link -> /Data/skd/work 33.75 TB GFI & SKD
users individual 20 TB GFI * *
projects group 4.25 TB GFI *
projects/metdata group 1 TB METDATA *
projects/farlab group 1 TB FARLAB *
projects/isomet group 0.25 TB ISOMET *
met group 15 TB MET
spengler group 15 TB SPENGLER
exprec group 4 TB EXPREC
metno met.no prognosis 1 TB GFI


GFI&SKD Storage folders inside /Data/skd/
Subfolder Purpose Size limit Group assign User quota Backup
share Symbolic link ->/Data/gfi/share/ - shared model data GFI & SKD
skd-share Several subfolders /Data/gfi/share/* -> /Data/skd/skd-share/. 51.25 TB GFI & SKD
scratch individual and group share 20 TB SKD
work shared short time (30days) 33.75 TB GFI & SKD
users individual 5 TB SKD * *
projects group 10 TB SKD *
Retrieved from "http://gfi.wiki.uib.no/index.php?title=GFI%26SKD_Storage&oldid=1083"