前情提要 话说上次DBA小倩通过删除home lv，把空间扩给了/分区，问题暂时得到了解决。
没过几天，领导找到小倩下达任务，客户说数据库在本地磁盘空间太小了又快要满了，由于之前用的服务器本地磁盘，性能也比较差，现在不想扩展服务器本地硬盘了。正好这台服务器有HBA卡，客户从别的高端存储上划了个2TB的LUN过来，映射给这台数据库服务器，想让DBA小倩来做一下数据迁移，当然停机时间己经客户己经安排好了，就差小倩上线了。
准备工作 正式停机之前还有一些准备工作要作，比如先把存储空间挂载到服务器上（后做也可以，用不了多久）。
检查一下当前磁盘空间情况，根目录满上要用光了。
1 2 3 4 5 6 7 8 9 [root@dbserver ~]# df -Th Filesystem Type Size Used Avail Use% Mounted on devtmpfs devtmpfs 1.9G 0 1.9G 0% /dev tmpfs tmpfs 1.9G 259M 1.6G 14% /dev/shm tmpfs tmpfs 1.9G 12M 1.9G 1% /run tmpfs tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/mapper/centos-root xfs 496G 493G 2.8G 100% / /dev/sda1 xfs 1014M 151M 864M 15% /boot tmpfs tmpfs 378M 0 378M 0% /run/user/0
再看一下磁盘情况，目前只是识别到了一块磁盘
1 2 3 4 5 6 7 8 [root@dbserver ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 500G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 499G 0 part ├─centos-root 253:0 0 495.1G 0 lvm / └─centos-swap 253:1 0 3.9G 0 lvm [SWAP] sr0 11:0 1 4.4G 0 rom
下面说一下怎么识别磁盘，正常这个操作是需要客户的系统管理员来操作（非必要DBA不要操作）。我这里用虚拟机给大家演示，就直接简化操作了（实际生产环境有可能还需要配置ISCSI、多路径，比较复杂，一般有专人处理，这个放在后面单独一章简单给大家做个知识补充 ）
我这里用虚拟机模拟，那么操作就是这样，扫描之后发现2T磁盘
1 2 3 4 5 6 7 8 9 10 11 12 13 14 [root@dbserver scsi_host]# lsscsi [0:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda [2:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0 [root@dbserver scsi_host]# echo "- - -" >/sys/class/scsi_host/host0/scan [root@dbserver scsi_host]# [root@dbserver scsi_host]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 500G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 499G 0 part ├─centos-root 253:0 0 495.1G 0 lvm / └─centos-swap 253:1 0 3.9G 0 lvm [SWAP] sdb 8:16 0 2T 0 disk sr0 11:0 1 4.4G 0 rom
下面对2T磁盘进行分区，并标记为LVM格式（划分区再标记这是标准做法，这样避免以后被别人当成空盘，直接给用了。当然也可以直接pvcreate /dev/sdb，但是这样就没有标记了。）
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [root@dbserver ~]# parted -s /dev/sdb mklabel gpt [root@dbserver ~]# parted -s /dev/sdb mkpart primary 2048s 100% [root@dbserver ~]# parted -s /dev/sdb toggle 1 lvm [root@dbserver ~]# fdisk -l /dev/sdb WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion. Disk /dev/sdb: 2199.0 GB, 2199023255552 bytes, 4294967296 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: gpt Disk identifier: 96DE5EB7-29A5-42AA-9651-03995AF95DEC # Start End Size Type Name 1 2048 4294965247 2T Linux LVM primary
下一步就是创建lvm，创建挂载点，挂载到操作系统上。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 [root@dbserver ~]# pvcreate /dev/sdb1 Physical volume "/dev/sdb1" successfully created. [root@dbserver ~]# vgcreate oradatavg /dev/sdb1 Volume group "oradatavg" successfully created [root@dbserver ~]# lvcreate -l 100%FREE -n oralv oradatavg Logical volume "oralv" created. [root@dbserver ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert root centos -wi-ao---- 495.12g swap centos -wi-ao---- <3.88g oralv oradatavg -wi-a----- <2.00t [root@dbserver ~]# vgs VG #PV #LV #SN Attr VSize VFree centos 1 2 0 wz--n- <499.00g 0 oradatavg 1 1 0 wz--n- <2.00t 0 [root@dbserver ~]# mkfs.xfs /dev/mapper/oradatavg-oralv meta-data=/dev/mapper/oradatavg-oralv isize=512 agcount=4, agsize=134217472 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=536869888, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=262143, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
开始迁移 开始停机，先让客户结束业务办理，把应用系统都关了，然后建议是先把LOCAL=NO进程全KILL，不然生产库一般很难正常shutdown immediate。操作步骤：先杀进程、关监听、关库、关EM
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 ps -ef |grep LOCAL=NO|grep -v grep|cut -c 9-15|xargs kill -9 [oracle@dbserver ~]$ lsnrctl stop LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 11-SEP-2024 23:07:17 Copyright (c) 1991, 2013, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) The command completed successfully [oracle@dbserver ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Wed Sep 11 23:07:00 2024 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL> shutdown immediate; Database closed. Database dismounted. ORACLE instance shut down. SQL> exit Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options [oracle@dbserver ~]$ emctl stop dbconsole OC4J Configuration issue. /u01/app/oracle/product/11.2.0/dbhome_1/oc4j/j2ee/OC4J_DBConsole_dbserver_oracle not found.
操作步骤是这样：
1、先重命名/u01目录为/u01.old
2、创建新的/u01目录，然后挂载/dev/mapper/oradatavg-oralv到/u01
3、给/u01目录赋予oracle:oinstall权限
4、使用rsync迁移/u01.old目录里的文件到/u01目录下（这里写成脚本用nohup运行，防止终端断开），这里注意rsync源目录后面要带/，就是比如这样/u01.old/
5、启动数据库恢复业务
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 [root@dbserver ~]#cd / [root@dbserver /]#mv /u01 /u01.old [root@dbserver /]# mkdir -p /u01 [root@dbserver /]# echo "/dev/mapper/oradatavg-oralv /u01 xfs defaults 0 0" >> /etc/fstab [root@dbserver /]# mount -a [root@dbserver /]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 1.9G 12M 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/mapper/centos-root 496G 493G 2.8G 100% / /dev/sda1 1014M 151M 864M 15% /boot tmpfs 378M 0 378M 0% /run/user/0 /dev/mapper/oradatavg-oralv 2.0T 33M 2.0T 1% /u01 [root@dbserver /]# chown oracle:oinstall /u01 [root@dbserver /]# echo "/usr/bin/rsync -av /u01.old/ /u01" >rsync.sh [root@dbserver /]# chmod +x rsync.sh [root@dbserver /]# nohup ./rsync.sh & #如果需要查看同步情况,最后显示完成 [root@dbserver /]# tail -f nohup.out app/oracle/product/11.2.0/dbhome_1/xdk/mesg/ app/oracle/product/11.2.0/dbhome_1/xdk/mesg/lpxus.msb app/oracle/product/11.2.0/dbhome_1/xdk/mesg/lpxus.msg app/oracle/product/11.2.0/dbhome_1/xdk/mesg/lpxzhs.msb app/oracle/product/11.2.0/dbhome_1/xdk/mesg/lsxus.msb app/oracle/product/11.2.0/dbhome_1/xdk/mesg/lsxus.msg app/oracle/product/11.2.0/dbhome_1/xdk/mesg/lsxzhs.msb sent 6,049,894,594 bytes received 709,411 bytes 145,797,686.87 bytes/sec total size is 6,046,010,967 speedup is 1.00
最后启动数据库，成功运行后，确认无错误，就可以把/u01.old目录删除释放空间了。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 [oracle@dbserver ~]$ lsnrctl start LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 11-SEP-2024 23:29:57 Copyright (c) 1991, 2013, Oracle. All rights reserved. Starting /u01/app/oracle/product/11.2.0/dbhome_1/bin/tnslsnr: please wait... TNSLSNR for Linux: Version 11.2.0.4.0 - Production System parameter file is /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora Log messages written to /u01/app/oracle/diag/tnslsnr/dbserver/listener/alert/log.xml Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=dbserver)(PORT=1521))) Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) STATUS of the LISTENER ------------------------ Alias LISTENER Version TNSLSNR for Linux: Version 11.2.0.4.0 - Production Start Date 11-SEP-2024 23:29:57 Uptime 0 days 0 hr. 0 min. 0 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora Listener Log File /u01/app/oracle/diag/tnslsnr/dbserver/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=dbserver)(PORT=1521))) The listener supports no services The command completed successfully [oracle@dbserver ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Wed Sep 11 23:30:03 2024 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to an idle instance. SQL> startup; ORACLE instance started. Total System Global Area 1586708480 bytes Fixed Size 2253624 bytes Variable Size 989859016 bytes Database Buffers 587202560 bytes Redo Buffers 7393280 bytes Database mounted. Database opened. SQL> exit
扩展知识 1、如果生产服务器用的是hba卡的话一般都2块卡，需要重新扫描一下hba卡就可以发现新硬盘了，我给大家看一下我这有一个生产环境样例是这样：主机有2块HBA卡为host1、host2。host0是本地的硬盘控制器。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 [root@rac1]# cd /sys/class/fc_host/ [root@rac1 fc_host]# ls -l total 0 lrwxrwxrwx 1 root root 0 Aug 6 03:02 host1 -> ../../devices/pci0000:00/0000:00:03.0/0000:04:00.0/host1/fc_host/host1 lrwxrwxrwx 1 root root 0 Aug 6 03:02 host2 -> ../../devices/pci0000:40/0000:40:02.2/0000:47:00.0/host2/fc_host/host2 [root@rac1 fc_host]# cd /sys/class/scsi_host/ [root@rac1 scsi_host]# ls -l total 0 lrwxrwxrwx 1 root root 0 Aug 6 03:02 host0 -> ../../devices/pci0000:00/0000:00:02.0/0000:02:00.0/host0/scsi_host/host0 lrwxrwxrwx 1 root root 0 Aug 6 03:02 host1 -> ../../devices/pci0000:00/0000:00:03.0/0000:04:00.0/host1/scsi_host/host1 lrwxrwxrwx 1 root root 0 Aug 6 03:02 host2 -> ../../devices/pci0000:40/0000:40:02.2/0000:47:00.0/host2/scsi_host/host2 [root@rac1 scsi_host]# lsscsi [0:0:0:0] disk HP LOGICAL VOLUME 4.04 /dev/sda [0:3:0:0] storage HP P830i 4.04 - [1:0:0:0] disk 3PARdata VV 3315 /dev/sdb [1:0:0:1] disk 3PARdata VV 3315 /dev/sdc [1:0:0:2] disk 3PARdata VV 3315 /dev/sdd [1:0:0:3] disk 3PARdata VV 3315 /dev/sde [1:0:0:4] disk 3PARdata VV 3315 /dev/sdf [1:0:0:254] enclosu 3PARdata SES 3315 - [1:0:1:0] disk 3PARdata VV 3315 /dev/sdg [1:0:1:1] disk 3PARdata VV 3315 /dev/sdh [1:0:1:3] disk 3PARdata VV 3315 /dev/sdi [1:0:1:4] disk 3PARdata VV 3315 /dev/sdj [1:0:1:5] disk 3PARdata VV 3315 /dev/sdk [1:0:1:254] enclosu 3PARdata SES 3315 - [2:0:0:0] disk 3PARdata VV 3315 /dev/sdl [2:0:0:1] disk 3PARdata VV 3315 /dev/sdm [2:0:0:3] disk 3PARdata VV 3315 /dev/sdn [2:0:0:4] disk 3PARdata VV 3315 /dev/sdo [2:0:0:5] disk 3PARdata VV 3315 /dev/sdp [2:0:0:254] enclosu 3PARdata SES 3315 - [root@rac1 scsi_host]#
那么需要进行的操作就是扫描两块HBA卡。
1 2 echo "- - -" >/sys/class/scsi_host/host1/scan echo "- - -" >/sys/class/scsi_host/host2/scan
2、如果是用万兆ISCSI连接的存储，那么就需要配置ISCSI，主机需要安装iscsi-initiator。生产一般会接2个网卡做为冗余，假设主机这边对接ISCSI存储的IP是192.168.1.2，存储侧是192.168.1.101、192.168.1.102
1 2 3 4 5 6 iscsiadm -m discovery -t st -p 192.168.1.101 iscsiadm -m node -p 192.168.1.101 -l iscsiadm -m discovery -t st -p 192.168.1.102 iscsiadm -m node -p 192.168.1.102 -l systemctl enable iscsi.service iscsiadm -m node -o update -n node.startup -v automatic
3、上面无论是FC-HBA或是ISCSI连接完存储后，剩下需要做的就是配置多路径multipath或者直接安装存储厂商的多路径软件（如果客户现场没有多个存储要同时用的话，推荐使用存储厂商的多路径软件，这样几乎不用啥配置），此处略长，大家要有兴趣我回头再写一篇。
未完待续 解决了这个问题，又来了一个棘手的活， 这回客户也是数据库装在根目录，然后根目录总满了，目前是一直手动清日志保证业务能用。但不一样的是，这次客户用的是虚拟机系统，而且虚拟机没用LVM装的系统，客户系统又不能停机，看看下期DBA小倩如何处理。
欢迎联系我一起讨论。我的微信号：Eric_xu_2023
也欢迎关注我的公众号：
