Linux 15th day: (August 29) Linux Advanced File System Management

Linux 15th day: (August 29) Linux Advanced File System Management

Chapter Content
Set File system quotas
Setting up and managing soft raid devices
Configuring logical Volumes
Setting LVM Snapshots
Btrfs File System

Configuring the quota system
Review
Executing in the kernel
Enabled as a file system
Different policies for different groups or users
Restrict by block or node
Execute soft limit (soft limit)
Hard Limit
Initialization
Partition mount options: Usrquota, Grpquota
Initialize database: Quotacheck

Set quotas for users
Perform
Open or Cancel quotas: Quotaon, Quotaoff
Direct Edit quota: Edquota username
Edit directly in the shell:
Setquota usename 4096 5120 50/foo
Define the original Standard user
Edquota-p user1 User2

Reporting Quota Status
Report
User survey: Quota
Quota Overview: Repquota
Other tools: Warnquota

What is raid
Raid:redundant Arrays of Inexpensive (independent) Disks
1988 by the University of California, Berkeley (University of California-berkeley) "A Case for redundant Arrays of inexpensive Disks".
Multiple disks to synthesize an "array" to provide better performance, redundancy, or both

Raid
Improved IO capability:
Parallel disk Read and write
Increased durability;
Disk redundancy to achieve
Level: Multiple disk organizations work together in different ways
How the raid is implemented:
External disk array: Adapter capability with expansion cards
On-chip RAID: Motherboard integrated RAID controller
Configure in BIOS before installing OS
Software RAID:

RAID level
RAID-0: Striped Roll, strip
RAID-1: Mirrored volume, mirror
RAID-2
..
RAID-5
RAID-6
RAID-10
RAID-01

RAID-0:
Read and write performance improvement;
Free space: n*min (S1,s2,...)
No fault-tolerant capability
Minimum number of disks: 2, 2

RAID-1:
Read performance improvement, write performance slightly decreased;
Free space: 1*min (S1,s2,...)
Have redundancy capability
Minimum number of disks: 2, 2N

RAID-4:
Multiple data disk XOR or operation values, stored in a dedicated check disk

RAID-5:
Improved read and write performance
Free space: (N-1) *min (s1,s2,...)
Fault tolerance: Allow up to 1 disks to be damaged
Minimum number of disks: 3,

RAID-6:
Improved read and write performance
Free space: (N-2) *min (s1,s2,...)
Fault tolerance: Allow up to 2 disks to be damaged
Minimum number of disks: 4, 4+

RAID Mix Type level

RAID-10:
Improved read and write performance
Free space: n*min (S1,s2,...) /2
Fault tolerance: Each group of mirrors can only have a bad piece
Minimum number of disks: 4, 4+

RAID-01, RAID-50

RAID7: Can be understood as an independent storage computer with operating system and management tools, can run independently, theoretically the highest performance of the RAID mode

Jbod:just a Bunch of Disks
Function: Combines the space of multiple disks into one large contiguous space for use
Free space: sum (s1,s2,...)

Common levels: RAID-0, RAID-1, RAID-5, RAID-10, RAID-50, JBOD

Soft raid
MDADM: Provides a management interface for soft raid
Add redundancy to a spare disk
Combined with MD (multi devices) in the kernel
RAID devices can be named/dev/md0,/dev/md1,/DEV/MD2,/dev/md3, and more

Implementation of software RAID
Mdadm: a modular tool
Syntax format for command: Mdadm[mode] <raiddevice> [options] <component-devices>
Supported RAID levels: LINEAR, RAID0, RAID1, RAID4, RAID5, RAID6, RAID10
Mode:
Create:-C
Assembly:-A
Monitoring:-F
Management:-F,-R,-a
<raiddevice>:/dev/md#
<component-devices>: any block device

-C: Create pattern
-N #: Create this raid with # blocks of devices;
-L #: Indicates the level of RAID to be created;
-A {Yes|no}: Automatically create device files for target RAID devices;
-C Chunk_size: Indicates the block size;
-X #: Indicates the number of free disks;
Example: Create a RAID5 of 10G free space
-D: Displays details of the raid;
Mdadm-d/dev/md#
Management mode:
-F: flag specifies that the disk is corrupt
-A: Adding disks
-R: Remove disk
Observe the status of MD:
Cat/proc/mdstat
To stop the MD device:
Mdadm-s/dev/md#

Soft RAID Configuration Instance
Create and define a RAID device using Mdadm
#mdadm-C/dev/md0-a yes-l 5-n 3-x 1/dev/sdb1/dev/sdc1/dev/sdd1/dev/sde1
Format each RAID device with a file system
#mke2fs-j/dev/md0
Testing RAID Devices
Mdadm allows checking the status of RAID devices
#mdadm--detail| D/dev/md0
Add a new Member
#mdadm –g/dev/md0–n4-a/DEV/SDF1

Soft RAID Testing and repair
Analog disk failure
#mdadm/dev/md0-f/dev/sda1
Remove a disk
#mdadm/dev/md0–r/dev/sda1
Repairing disk failures from software RAID disks
Replace the failed disk and boot
Rebuilding partitions on an alternate drive
#mdadm/dev/md0-a/dev/sda1
Mdadm,/proc/mdstat, and system log information

Soft RAID Management
Build configuration file: Mdadm–d–s >>/etc/mdadm.conf
Stop service: mdadm–s/dev/md0
Activation: mdadm–a–s/dev/md0 activation
Delete Raid information: MDADM–ZERO-SUPERBLOCK/DEV/SDB1

Logical Volume Manager (LVM)
A layer of abstraction that allows for easy operation of the volume, including resizing the file system
Allows file systems to be re-organized across multiple physical devices
To designate a device as a physical volume
Create a volume group with one or more physical volumes
A physical volume is defined by a fixed-size physical region (physical extent,pe).
A logical volume created on a physical volume is made up of a physical region (PE)
You can create a file system on a logical volume

Introduction to LVM
Lvm:logical Volume Manager,version:2
Dm:device Mapper: A module that organizes one or more underlying block devices into a single logical device
Device Name:/dev/dm-#
Soft Links:
/dev/mapper/vg_name-lv_name
/dev/mapper/vol0-root
/dev/vg_name/lv_name
/dev/vol0/root

LVM changes the file system's capacity
LVM can flexibly change the capacity of LVM
Exchange of PE for data conversion, the original LV PE transferred to other devices to reduce the LV capacity, or other equipment in the PE added to the LV to increase capacity

View the usage of LVM
Click "System", "Management", "Logical Volume Manager"

Increase or decrease a logical volume
After opening the logical Volume manager, click "Edit Properties" to open the LVM Properties dialog box:

Delete a logical volume
Deleting a logical volume must first delete the LV, then delete the VG, and finally remove the PV
Click the LV logical volume for the logical view, Volume group, logical Volume Manager
Click "Remove selected Logical Volume", delete VG, and finally delete PV.

PV Management Tools
Display PV Information
PVS: Brief PV information display
Pvdisplay
Create PV
Pvcreate/dev/device
Show Volume Group
VGs
Vgdisplay
Create a volume group
vgcreate [-S #[kkmmggttppee]] volumegroupname Physicaldevicepath [Physicaldevicepath ...]
Manage Volume groups
Vgextend volumegroupname Physicaldevicepath [Physicaldevicepath ...]
Vgreduce volumegroupname Physicaldevicepath [Physicaldevicepath ...]
Delete a volume group
Do pvmove first, then do Vgremove

LV Management Tools
displaying logical volumes
Lvs
Lvdisplay
Creating a logical Volume
Lvcreate-l #[mmggtt]-n NAME volumegroup
Delete a logical volume
Lvremove/dev/vg_name/lv_name
Resetting file system size
Fsadm[options] Resize device [New_size[bkmgtep]
RESIZE2FS [-F] [-f] [-m] [-p] [-p] device [new_size]

Extending and shrinking logical volumes
To extend a logical volume:
# lvextend-l [+]#[MMGGTT]/dev/vg_name/lv_name
# Resize2fs/dev/vg_name/lv_name
To shrink a logical volume:
# Umount/dev/vg_name/lv_name
# e2fsck-f/dev/vg_name/lv_name
# Resize2fs/dev/vg_name/lv_name #[mmggtt]
# lvreduce-l [-]#[MMGGTT]/dev/vg_name/lv_name
# mount

Creating a logical Volume instance
Create a physical volume
Pvcreate/dev/sda3
Assigning a physical volume to a volume group
Vgcreate Vg0/dev/sda3
To create a logical volume from a volume group
Lvcreate-l 256m-n Data vg0
Mke2fs-j/dev/vg0/data
Mount/dev/vg0/data/mnt/data

Logical Volume Manager Snapshot
A snapshot is a special logical volume that is an exact copy of a logical volume that exists when a snapshot is generated
Snapshots are the most appropriate choice for temporary copies of existing datasets and other operations that require backup or replication.
Snapshots consume space only when they are different from the original logical volume.
A certain amount of space is allocated to a snapshot when it is generated, but only if the original logical volume or snapshot has changed to use the space
When the original logical volume changes, the old data is copied to the snapshot.
The snapshot contains only the changed data from the original logical volume or the data changed in the snapshot since the snapshot was generated
The volume size of a snapshot requires only the 15%~20% of the original logical volume. You can also use Lvextend to zoom in on a snapshot.
The snapshot is to record the system information at that time, as if there were any data changes in the future, the original data will be moved to the snapshot area, the unchanged area is shared by the snapshot area and the file system.
Since the snapshot area and the original LV share a lot of PE chunks, so the snapshot to be snapshot with the LV must be on the same VG! When the system recovers, the number of files cannot be higher than the actual capacity of the snapshot area.

Using LVM Snapshots
To create a snapshot of an existing logical volume
#lvcreate-L 64-s-N snap-data-p r/dev/vg0/data
Mount Snapshot
#mkdir-P/mnt/snap
#mount-O Ro/dev/vg0/snap-data/mnt/snap
Deleting a snapshot
#umount/mnt/databackup
#lvremove/dev/vg0/databackup

Btrfs File System
Technical Preview Version
Btrfs (B-tree, Butter FS, Better FS), GPL, Oracle, CoW
Core Features:
Multi-Physical volume support: Btrfs can be composed of multiple underlying physical volumes, RAID support, and online "add", "Remove", "modify"
Copy-on-write update mechanism (CoW): Copy, update, and replace pointers instead of "in-place" updates
Data and meta-data check code: Checksum
Sub-Volume: Sub_volume
Snapshots: Snapshots that support snapshots
Transparent compression

Btrfs File System
File System creation:
Mkfs.btrfs
-L ' LABEL '
-D <type>: Raid0, RAID1, RAID5, Raid6, RAID10, single
-M <profile>: RAID0, RAID1, RAID5, Raid6, RAID10, single, DUP
-O <feature>
-O List-all: Lists all supported feature
Mkfs.btrfs-l mydata-f/DEV/SDB/DEV/SDC
Properties View:
Btrfsfilesystem Show; Blkid
Btrfs filesystem Show–mounted|all-devices
Mount File System:
Mount-t Btrfs/dev/sdbmount_point
Transparent compressor System:
Mount-o Compress={lzo|zlib} DEVICE mount_point
Modify File System size online
Man Btrfs
Btrfsfilesystem Resize-10g/mydata
Btrfsfilesystem Resize +5g/mydata
Btrfsfilesystem Resize Max/mydata
View
DF–LH; Btrfsfilesytemdf/mydata
Add Device: Man Btrfs-device
Btrfsdevice Add/dev/sdd/mydata
Balance data: Man btrfs-banlance
Btrfs banlance Status/mydata
Btrfs banlance Start/mydata
Btrfs banlance Pause/mydata
Btrfs banlance Cancel/mydata
Btrfs banlance Resume/mydata
Remove device
Btrfs Device Delete/dev/sdb/mydata
Modify RAID Level: Note the raid requirements for the number of members
Btrfs Balance Start-mconvert=raid1|raid0|raid5/mydata
Btrfs Balance Start-dconvert=raid1|raid0|raid5/mydata
Sub-volume management: Man Btrfs-subvolume
Btrfssubvolumelist/mydata to view information such as the sub-volume ID
Btrfssubvolumecreate/mydata/subv1
Umount/mydata
Mount-o SUBVOL=SUBV1/DEV/SDD/MNT/SUBV1
Btrfssubvolumeshow/mnt/subv1
Mount/dev/sdb/mydata Hanging Parent volume, sub-volume auto mount
Mount-o subvolid=###/dev/sdd/mnt/subv1
Sub-volume management
Btrfssubvolumeshow/mnt/subv1
Btrfssubvolumedelete/mydata/subv1
To create a snapshot:
Btrfssubvolumesnapshot/mydata/subv1 \/MYDATA/SNAPSHOT_SUBV1
Btrfssubvolumelist/mydata
Deleting a snapshot
Btrfssubvolumedelete/mydata/snapshot_subv1
Take a snapshot of a file (current volume)
Cd/mydata/subv1
Cp--reflinktestfilesnapshot_testfile

Linux 15th day: (August 29) Linux Advanced File System Management