User Tools

Site Tools


warewulf_worker_node_images

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
warewulf_worker_node_images [2020/06/26 20:54]
deadline created
warewulf_worker_node_images [2021/05/20 17:18] (current)
brandonm [Updating Limulus Node VNFS Images] Word, spelling, formatting, hyphenation, and phrasing fixes
Line 1: Line 1:
 =====Warewulf Worker Node Images===== =====Warewulf Worker Node Images=====
  
-All Limulus HPC system use the [[https://warewulf.lbl.gov/|Warewulf Toolkit]] to manage worker node image. The Warewulf toolkit allows worker nodes to boot "disk-less" using a RAM disk. The RAM disk and bootable kernel are managed by Warewulf. The following is a description of how Warewulf is configured and run on Limulus systems. Consult the Warewulf site for more detailed information. +All Limulus HPC systems use the [[https://warewulf.lbl.gov/|Warewulf Toolkit]] to manage worker node images. The Warewulf toolkit allows worker nodes to boot "disk-less" using a RAM disk. The RAM disk and bootable kernel are managed by Warewulf. The following is a description of how Warewulf is configured and run on Limulus systems. Consult the Warewulf site for more detailed information. 
  
 Note: Data Analytics systems use [[https://ambari.apache.org|Apache Ambari]] to manage and install images to local disk on the worker nodes. This section does not apply to Data Analytics Systems. Note: Data Analytics systems use [[https://ambari.apache.org|Apache Ambari]] to manage and install images to local disk on the worker nodes. This section does not apply to Data Analytics Systems.
Line 7: Line 7:
 ====Using Warewulf (Quick Start)==== ====Using Warewulf (Quick Start)====
  
-Managing worker node images includes three main components:+Under normal circumstances, administrators will not need to manage node images. All HPC systems are configured to use default Limulus Warewulf images. The following background information will help with managing and updating worker node images. There are three main components:
  
-  - **File System Images** - Each worker node is provided a RAM disk file system image. This image contains all the "local" files for the node including configuration,  executables, and libraries. These file system is called a Virtual Network File Systems (VNFS) image. +  - **File System Images** - Each worker node is provided a RAM disk file system image. This image contains all the "local" files for the nodeincluding configuration, executables, and libraries. This file system is called a Virtual Network File System (VNFS) image. 
-  - **Boostrap Kernel** - kernel image that used to start the worker node. This image usually contains the same kernel version as the main node, but it is not required.  +  - **Boostrap Kernel** - kernel image used to start the worker node. This image usually contains the same kernel version as the main node, but it is not required to.  
-  - **Files** - these are generally configuration files that are maintained on the main node and sent to worker nodes when booting and then periodically as the node runs (if Warewulf notes any changes in the files). These files can be the same as the ones used on the head node or they can be specific to the worker nodes. +  - **Files** - These are generally configuration files that are maintained on the main node and sent to worker nodes when booting and then periodically as the node runs (if Warewulf notes any changes in the files). These files can be the same as the ones used on the headnode, or they can be specific to the worker nodes. 
  
-VNF images, bootstrap kernels,and files can all be loaded into the Warewulf database. See the "Modifying Limulus VNFS, Bootstrap, and System Filesbelow for mor information on this process.+VNFS images, bootstrap kernels, and files can all be loaded into the Warewulf database. See the [[warewulf_worker_node_images#Modifying Limulus VNFS, Bootstrap, and System Files]] section below for more information on this process.
  
-To view the installed **VNFS Images** use the ''wwsh'' shell command as follows. Some helpful tips: +To view the installed **VNFS Images**use the ''wwsh'' shell command as follows. Some helpful tips: 
-  * you start an Warewuf shell by not including any arguments +  * You can start a Warewulf shell by not including any arguments. 
-  * incomplete command to wwsh bring up a help screen+  * Incomplete arguments to wwsh bring up a help screen.
    
 <code> <code>
Line 28: Line 28:
 This example shows two VNFS images, //centos7.7// and //co7_base// This example shows two VNFS images, //centos7.7// and //co7_base//
  
-Similarly the available **bootstrap kernels** can be listed as follows.+Similarlythe available **bootstrap kernels** can be listed as follows.
  
 <code> <code>
Line 36: Line 36:
 </code> </code>
  
-**Node specific files** can be delivered to the nodes at boot time. The ''wwsh file list'' command can be used to show the current files that are available in the Warewulf database (these files must be specifically assigned using the ''wwsh provision'' command described below. For example, the default files available for the nodes are:+**Node-specific files** can be delivered to the nodes at boot time. The ''wwsh file list'' command can be used to show the current files that are available in the Warewulf database (these files must be specifically assigned using the ''wwsh provision'' command described below). For example, the default files available for the nodes are listed below. To accommodate new users, each node will check for an updated ''/etc/{passwd,group,shadow}'' on the main host. (Note that after any change is made to a local file that is part of the Warewulf database on the main node, a ''wwsh file sync'' command must be issued to update the file in the "files" database.)
  
 <code> <code>
Line 53: Line 53:
 </code> </code>
  
-The **node provisioning** can be viewed using the ''wwsh provision list'' command (i.e. a short listing of the VNFS, Bootstrap, and Files assigned to each node).+The **node provisioning** can be viewed using the ''wwsh provision list'' command. This is a short listing of the VNFS imagesbootstrap kernels, and files assigned to each node.
 <code> <code>
  
Line 64: Line 64:
 </code> </code>
  
-A much longer and detailed listing can be generated by using ''wwsh provision print'' command. If no essential image of file has been assigned (in this case below a VNFS imsgr) the listing will show ''UNDEF'' and the node will not be able to boot.+A much longer and detailed [[provision listing]] can be generated by using the ''wwsh provision print'' command. If no essential image or file has been assigned (in the case belowa VNFS image)the listing will show ''UNDEF'' and the node **will not be able to boot**.
  
 <code> <code>
Line 75: Line 75:
 </code> </code>
  
-The ''wwsh provision'' command is used to assign specific compenents (VNFS images, Bootstratp Kernels, and Files to nodes.  See below. +The ''wwsh provision'' command is used to assign specific components (VNFS images, bootstrap kernels, and files) to nodes.  See below. 
  
 ====Updating Limulus Node VNFS Images==== ====Updating Limulus Node VNFS Images====
  
-Limulus Computing provides Warewulf node images in an RPM package. These are easily installed and manages on the cluster. See below for instructions on how to modify these images. +Limulus Computing provides Warewulf node images in an RPM package. These are easily installed and managed on the cluster. See below for instructions on how to modify these images. 
  
 As configured, Limulus HPC systems have a basic node image that can be used to run the worker nodes.  As configured, Limulus HPC systems have a basic node image that can be used to run the worker nodes. 
-Form time to time Limulus Computing may update the Warewulf images. These can be installed by +From time to time Limulus Computing may update the Warewulf images. These can be installed by 
-first deleting the existing VNS and then using ''yum'' package manager as follows to install the new VNFS+first deleting the existing VNFS and then using the ''yum'' package manager as follows to install the new VNFS
 (Note: the update can take several minutes). (Note: the update can take several minutes).
  
 <code> <code>
-rpm -e vnfs-co7_base +rpm -e vnfs-co7_base
-</code> +
- +
-<code> +
 # yum install vnfs-co7_base # yum install vnfs-co7_base
 Loaded plugins: fastestmirror, langpacks Loaded plugins: fastestmirror, langpacks
Line 98: Line 94:
 </code> </code>
  
-The next step is to assign the new VNFS to the worker nodes. Assuming the new VNFS installed properly, the <tt>wwsh provision</tt> command can be used to set the VNFS image for the nodes.  (Use ''wwsh vnfs list'' to check that the VNFS image is available.) The following command sets the VNFS image for nodes n1, n1, n2. Enter ''Yes'' to the confirmation question. (Change the ''set -n[0-2]'' to ''set -n[0-6]'' for the eight nodes double wide systems.)+The next step is to assign the new VNFS to the worker nodes. Assuming the new VNFS installed properly, the ''wwsh provision'' command can be used to set the VNFS image for the nodes.  (Use ''wwsh vnfs list'' to check that the VNFS image is available.) The following command sets the VNFS image for nodes n0, n1, and n2. Enter ''Yes'' to the confirmation question. (Change the ''set -n[0-2]'' to ''set -n[0-6]'' for the eight-node double-wide systems.)
  
 <code> <code>
Line 109: Line 105:
 </code> </code>
  
-The current node provisioning can be checked by using the "list" option for ''wwsh provision'' command.+The current node provisioning can be checked by using the "list" option for the ''wwsh provision'' command.
  
 <code> <code>
Line 120: Line 116:
 </code> </code>
  
-The system should be ready to reuse the new images when the nodes are rebooted. To reboot all the nodes enter:+The system should be ready to use the new images when the nodes are rebooted. To reboot all nodes enter:
 <code> <code>
-pdsh reboot+pdsh reboot
 </code> </code>
  
-After a few minutes the nodes should be up and runing (check with ''wwtop'').+After a few minutes the nodes should be up and running (check with ''wwtop'').
  
 ====Updating Limulus Node Bootstrap Images==== ====Updating Limulus Node Bootstrap Images====
warewulf_worker_node_images.1593204879.txt.gz · Last modified: 2020/06/26 20:54 by deadline