Skip to content

13.2 Software Release

13.2.1 Overview

Software releaserefers to the process in which experiment software administrators publish software packages required by the computing environment. These packages are used by experiment users when submitting jobs.

The computing center usesCVMFS (CernVM File System)to host software for different experiment groups.

cvmfspublisher.ihep.ac.cnis the dedicated software publishing server. It:

  • Only allows login viaSSH private key authentication
  • Is restricted tosystem users
  • Provides an environment consistent with login nodes
  • DoesNOT include other network storage systems

This ensures that all compiled software: - Depends only on/cvmfs- Can run in an environment where only/cvmfsis mounted

13.2.2 Architecture

The software publishing workflow and architecture are shown below:

IHEP CVMFS

13.2.3 Software Release Workflow

1. Apply for Publishing Account

The software administrator should:

  • Contact the computing center
  • Request a publishing account
  • Provide:
  • Repository name:$repo
  • SSH public key

The CVMFS administrator will create the accountusername.

2. Generate SSH Key Pair

ssh-keygen

3. Login to Publisher Server

[user@localhost~]sshusername@cvmfspublisher.ihep.ac.cn-iyourprivatekey

4. Check Repository Status

[user@cvmfspublisher~]$cvmfs_serverlist|greprepositoryname|greptransaction

If the repository is in transaction state, it means another user is modifying it.

Coordinate with others or wait before proceeding.

5. Start Transaction

[user@cvmfspublisher~]$cvmfs_servertransactionrepositoryname

Notes: Maximum transaction time: 24 hours After expiration, publishing will fail with:token expired

13.2.4 Software Build Methods

There are three common methods for building software in a CVMFS repository:

a. Copy-Based Deployment

The software administrator compiles the software in an independent build environment.

After compilation, the generated artifacts are copied to the publisher server:

/cvmfs/repositoryname/...

b. Native Build on Publisher

The software administrator compiles the software directly on the cvmfspublisher.ihep.ac.cn server.

The compiled output is written directly to:

/cvmfs/repositoryname/...

To ensure compatibility with specific operating systems, use the container environment:

[user@cvmfspublisher~]$/cvmfs/container.ihep.ac.cn/bin/hep_containershellSL5/SL6/SL7
Inside the container:
Singularityyourimage.sif:~>Buildsoftwareinto:/cvmfs//dirAAll/cvmfs/*.ihep.ac.cnrepositoriesareaccessibleexit

Handling Build Failures

If any error occurs during the build process (e.g., timeout, failure), abort the current transaction:

cvmfs_serverabortrepositoryname
This will discard all changes made during the current transaction.

Then restart:

cvmfs_servertransactionrepositoryname

13.2.5 Publish Software

After completing the build, exit the repository directory and run:

cd/cvmfscvmfs_serverpublish-a"tagname"-m"tag description"repositoryname
6. List Recent Published Versions

cvmfs_servertag-lrepositorynameName│Revision│Channel│Timestamp│Branch│Description─────────────────────────────┼──────────┼─────────┼──────────────────────┼────────┼─────────────trunk-previous│1│trunk│13Jul202014:19:08││defaultundotargetgeneric-2020-07-13T06:19:07Z│1│trunk│13Jul202014:19:08││trunk│2│trunk│13Jul202015:40:53││currentHEADorigin│2│trunk│13Jul202015:40:53││create─────────────────────────────┴──────────┴─────────┴──────────────────────┴────────┴─────────────listingcontains4tags
7. Availability

Newly published software becomes available in ~10 minutes

8. Common Issues

Issue 1: Open File Handles Prevent Publishing

Error:WARNING!Thereareopenread-onlyfiledescriptors
Check processes:
lsof/cvmfs/repositoryname
Inspect:
strace-p
Kill process:
kill-9
Retry:
cvmfs_serverpublishrepositoryname

Software Access Workflow

Users access software via CVMFS client.

1.Install CVMFS Client
yum-yinstallhttps://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpmyum-yinstallcvmfscvmfs-config-defaultcvmfs_configsetup
2. Configure /etc/cvmfs/default.local 1. Install CVMFS Client

yum-yinstallhttps://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpmyum-yinstallcvmfscvmfs-config-defaultcvmfs_configsetup
2. Configure /etc/cvmfs/default.local
CVMFS_REPOSITORIES=bes3.ihep.ac.cn,common.ihep.ac.cn,cvmfs-config.cern.chCVMFS_CACHE_BASE=/scratch/cvmfsCVMFS_QUOTA_LIMIT=40960CVMFS_HTTP_PROXY=DIRECTORCVMFS_HTTP_PROXY="http://proxyIP:3128"
3. Disable SELinux
setenforce0
4. Configure Stratum Server
curl-o/etc/cvmfs/domain.d/cern.ch.local\http://cvmfs-stratum-one.ihep.ac.cn/cvmfs/software/client_configure/cern.ch.localln-s/etc/cvmfs/domain.d/cern.ch.local/etc/cvmfs/domain.d/egi.eu.localln-s/etc/cvmfs/domain.d/cern.ch.local/etc/cvmfs/domain.d/opensciencegrid.org.local

5. Configure IHEP Domain and Keys

bash mkdir -p /etc/cvmfs/keys/ihep.ac.cn curl -o /etc/cvmfs/keys/ihep.ac.cn/ihep.ac.cn.pub \ http://cvmfs-stratum-one.ihep.ac.cn/cvmfs/software/client_configure/ihep.ac.cn/ihep.ac.cn.pub curl -o /etc/cvmfs/domain.d/ihep.ac.cn.conf \ http://cvmfs-stratum-one.ihep.ac.cn/cvmfs/software/client_configure/ihep.ac.cn.conf6. Verify Installationbash cvmfs_config probe

One-Click Install Script

Save the following asinstall_cvmfs_client.shand run as root:

#!/bin/bash# === CVMFS client one-click install script (IHEP) ===# For RHEL/CentOS, requires root privileges# Usage: sudo bash install_cvmfs_client.shset-eecho"====[1/7] Install CVMFS packages ===="yum-yinstallhttps://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpmyum-yinstallcvmfscvmfs-config-defaultcvmfs_configsetupecho"====[2/7] Configure default.local ===="cat>/etc/cvmfs/default.local<<'EOF'CVMFS_REPOSITORIES=bes3.ihep.ac.cn,common.ihep.ac.cn,cvmfs-config.cern.chCVMFS_CACHE_BASE=/scratch/cvmfsCVMFS_QUOTA_LIMIT=40960CVMFS_HTTP_PROXY=DIRECTEOFecho"====[3/7] Disable SELinux enforce ===="setenforce0echo"====[4/7] Configure Stratum server ===="curl-o/etc/cvmfs/domain.d/cern.ch.localhttp://cvmfs-stratum-one.ihep.ac.cn/cvmfs/software/client_configure/cern.ch.localln-sf/etc/cvmfs/domain.d/cern.ch.local/etc/cvmfs/domain.d/egi.eu.localln-sf/etc/cvmfs/domain.d/cern.ch.local/etc/cvmfs/domain.d/opensciencegrid.org.localecho"====[5/7] Configure IHEP domain and keys ===="mkdir-p/etc/cvmfs/keys/ihep.ac.cncurl-o/etc/cvmfs/keys/ihep.ac.cn/ihep.ac.cn.pubhttp://cvmfs-stratum-one.ihep.ac.cn/cvmfs/software/client_configure/ihep.ac.cn/ihep.ac.cn.pubcurl-o/etc/cvmfs/domain.d/ihep.ac.cn.confhttp://cvmfs-stratum-one.ihep.ac.cn/cvmfs/software/client_configure/ihep.ac.cn.confecho"====[6/7] Verify CVMFS client ===="cvmfs_configprobeecho"====[7/7] Done ===="echo"CVMFS client installed. Access software under /cvmfs."

Note: To use a proxy, modifyCVMFS_HTTP_PROXYin/etc/cvmfs/default.local, e.g.:

CVMFS_HTTP_PROXY="http://202.122.33.47:3128;http://202.122.33.52:3128"

AI 助手
快速提问 ×

有问题?输入后跳转到对话页继续交流

Baidu
map