Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Thursday, 16 November 2023

Adding a device to a raid with mdadm

Reordering partitions to make room

So my current hard drives are:

# lsscsi
[0:0:0:0]    disk    ATA      Samsung SSD 860  1B6Q  /dev/sda 
[4:0:0:0]    disk    ATA      WDC WD4003FZEX-0 1A01  /dev/sdb 
[5:0:0:0]    disk    ATA      WDC WD5000AAKS-0 3B01  /dev/sdc 
[5:0:1:0]    disk    ATA      ST2000DM001-1CH1 CC29  /dev/sdd 
[10:0:0:0]   disk    WD       Ext HDD 1021     2021  /dev/sde 
[11:0:0:0]   disk    WD       Ext HDD 1021     2021  /dev/sdf 

I had a lot of unused space on /dev/sdb, and thought I could rearrange the partitions and use it as an additional device in my (software) RAID.

I used Parted1, but GParted2 is also very nice. GParted can also scan my /dev/md127, which is a raid device.

Of course there's the general problem nowadays of looking what kind of MBR/GPT or whatever boot schema you use.

So:

Disk /dev/sdb: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: WDC WD4003FZEX-0
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: D344E388-FA66-4B30-BA00-4D3B12A07B79
Device          Start        End    Sectors  Size Type
/dev/sdb1        2048       4095       2048    1M BIOS boot
/dev/sdb2        4096    1028095    1024000  500M Linux filesystem
/dev/sdb3     1028096 5273437500 5272409405  2.5T Linux filesystem
/dev/sdb4  5273438208 7420921855 2147483648    1T Linux filesystem

So it's a 4 TB hard drive, and it turns out that I only use /dev/sdb3. (The other partitions are from an old Linux install before I got a nice SDD and reinstalled Linux/Windows on that.)

So I could first rearrange it: resize /dev/sdb3 to 2T and move it to /dev/sdb1.

That worked fine using Parted. It just took some time to actually move all the data.

And then create a new /dev/sdb2 of 2T for my raid with gdisk3 (the GPT version of fdisk).

I had to look up the appropriate partition type in GPT4 for mdadm.

# gdisk /dev/sdb
GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Command (? for help): n
Partition number (2-128, default 2): 
First sector (34-7814037134, default = 3907006464) or {+-}size{KMGTP}: 
Last sector (3907006464-7814037134, default = 7814035455) or {+-}size{KMGTP}: 
Current type is 8300 (Linux filesystem)


Hex code or GUID (L to show codes, Enter = 8300): fd00
Changed type of partition to 'Linux RAID'

Command (? for help): p
Disk /dev/sdb: 7814037168 sectors, 3.6 TiB
Model: WDC WD4003FZEX-0
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): D344E388-FA66-4B30-BA00-4D3B12A07B79
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 3693 sectors (1.8 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      3907006463   1.8 TiB     8300  
   2      3907006464      7814035455   1.8 TiB     FD00  Linux RAID

Adding the device

Current raid setup:

# mdadm --assemble /dev/md127 /dev/sdd1 /dev/sde1 /dev/sdf1
mdadm: /dev/md127 has been started with 3 drives.

# mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Wed Mar  6 22:16:05 2013
        Raid Level : raid1
        Array Size : 1953380160 (1862.89 GiB 2000.26 GB)
     Used Dev Size : 1953380160 (1862.89 GiB 2000.26 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

       Update Time : Wed Aug 16 09:08:07 2023
             State : clean 
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : micemouse:0
              UUID : ed4531c4:59c132b2:a6bfc3d1:6da3b928
            Events : 7285

    Number   Major   Minor   RaidDevice State
       4       8       65        0      active sync   /dev/sde1
       5       8       81        1      active sync   /dev/sdf1
       3       8       49        2      active sync   /dev/sdd1
# cat /proc/mdstat
Personalities : [raid1] 
md127 : active raid1 sde1[4] sdd1[3] sdf1[5]
      1953380160 blocks super 1.2 [3/3] [UUU]
      
unused devices: <none>

Adding a device:

mdadm --add /dev/md127 /dev/sdb2
mdadm: added /dev/sdb2
  
md127 : active raid1 sdb2[6](S) sde1[4] sdd1[3] sdf1[5]
      1953380160 blocks super 1.2 [3/3] [UUU]
      
unused devices: <none>
 mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Wed Mar  6 22:16:05 2013
        Raid Level : raid1
        Array Size : 1953380160 (1862.89 GiB 2000.26 GB)
     Used Dev Size : 1953380160 (1862.89 GiB 2000.26 GB)
      Raid Devices : 3
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Sat Aug 19 16:47:58 2023
             State : clean 
    Active Devices : 3
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 1

Consistency Policy : resync

              Name : micemouse:0
              UUID : ed4531c4:59c132b2:a6bfc3d1:6da3b928
            Events : 7286

    Number   Major   Minor   RaidDevice State
       4       8       65        0      active sync   /dev/sde1
       5       8       81        1      active sync   /dev/sdf1
       3       8       49        2      active sync   /dev/sdd1

       6       8       18        -      spare   /dev/sdb2
[
root@localhost ~]# mdadm --grow /dev/md127 --raid-devices=4
raid_disks for /dev/md127 set to 4

  # mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Wed Mar  6 22:16:05 2013
        Raid Level : raid1
        Array Size : 1953380160 (1862.89 GiB 2000.26 GB)
     Used Dev Size : 1953380160 (1862.89 GiB 2000.26 GB)
      Raid Devices : 4
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Sat Aug 19 16:49:32 2023
             State : clean, degraded, recovering 
    Active Devices : 3
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 1

Consistency Policy : resync

    Rebuild Status : 0% complete

              Name : micemouse:0
              UUID : ed4531c4:59c132b2:a6bfc3d1:6da3b928
            Events : 7290

    Number   Major   Minor   RaidDevice State
       4       8       65        0      active sync   /dev/sde1
       5       8       81        1      active sync   /dev/sdf1
       3       8       49        2      active sync   /dev/sdd1
       6       8       18        3      spare rebuilding   /dev/sdb2

And now we wait.

# cat /proc/mdstat
Personalities : [raid1] 
md127 : active raid1 sdb2[6] sde1[4] sdd1[3] sdf1[5]
      1953380160 blocks super 1.2 [4/3] [UUU_]
      [>....................]  recovery =  0.1% (2956032/1953380160) finish=1156.0min speed=28118K/sec
      
unused devices: <none>

Ready in about 12 hours.

References

[1] Parted User’s Manual
https://www.gnu.org/software/parted/manual/parted.html
[2] GParted is a free partition editor for graphically managing your disk partitions.
https://gparted.org/
[3] gdisk(8) - Linux man page
https://linux.die.net/man/8/gdisk/
[4] Raid Wiki Kernel Org - Partition Types
https://raid.wiki.kernel.org/index.php/Partition_Types
WP Guru - How to add a drive to software RAID with mdadm
https://wpguru.co.uk/2021/01/expand-software-raid-mdadm/

Saturday, 13 May 2023

Mounting and umounting my MDADM disk array

In my dmesg, when connecting my USB harddrives, I get the following messages:

[ 3455.138687] usb 1-1.5: new high-speed USB device number 3 using ehci-pci
[ 3455.218450] usb 1-1.5: New USB device found, idVendor=1058, idProduct=1021, bcdDevice=20.21
[ 3455.218462] usb 1-1.5: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3455.218468] usb 1-1.5: Product: Ext HDD 1021
[ 3455.218472] usb 1-1.5: Manufacturer: Western Digital
[ 3455.218476] usb 1-1.5: SerialNumber: 574D43315431373033343231
[ 3455.260880] usb-storage 1-1.5:1.0: USB Mass Storage device detected
[ 3455.261279] scsi host10: usb-storage 1-1.5:1.0
[ 3455.261525] usbcore: registered new interface driver usb-storage
[ 3455.267542] usbcore: registered new interface driver uas
[ 3455.287643] usb 1-1.6: new high-speed USB device number 4 using ehci-pci
[ 3455.367335] usb 1-1.6: New USB device found, idVendor=1058, idProduct=1021, bcdDevice=20.21
[ 3455.367347] usb 1-1.6: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3455.367353] usb 1-1.6: Product: Ext HDD 1021
[ 3455.367357] usb 1-1.6: Manufacturer: Western Digital
[ 3455.367361] usb 1-1.6: SerialNumber: 574D43315432313138383334
[ 3455.367897] usb-storage 1-1.6:1.0: USB Mass Storage device detected
[ 3455.368175] scsi host11: usb-storage 1-1.6:1.0
[ 3456.301327] scsi 10:0:0:0: Direct-Access WD Ext HDD 1021 2021 PQ: 0 ANSI: 4
[ 3456.302077] sd 10:0:0:0: Attached scsi generic sg4 type 0
[ 3456.302737] sd 10:0:0:0: [sde] 3907024896 512-byte logical blocks: (2.00 TB/1.82 TiB)
[ 3456.304758] sd 10:0:0:0: [sde] Write Protect is off
[ 3456.304771] sd 10:0:0:0: [sde] Mode Sense: 17 00 10 08
[ 3456.306754] sd 10:0:0:0: [sde] No Caching mode page found
[ 3456.306763] sd 10:0:0:0: [sde] Assuming drive cache: write through
[ 3456.335251] sde: sde1
[ 3456.335455] sd 10:0:0:0: [sde] Attached SCSI disk
[ 3456.428905] scsi 11:0:0:0: Direct-Access WD Ext HDD 1021 2021 PQ: 0 ANSI: 4
[ 3456.430291] sd 11:0:0:0: Attached scsi generic sg5 type 0
[ 3456.431366] sd 11:0:0:0: [sdf] 3907024896 512-byte logical blocks: (2.00 TB/1.82 TiB)
[ 3456.433375] sd 11:0:0:0: [sdf] Write Protect is off
[ 3456.433387] sd 11:0:0:0: [sdf] Mode Sense: 17 00 10 08
[ 3456.435502] sd 11:0:0:0: [sdf] No Caching mode page found
[ 3456.435515] sd 11:0:0:0: [sdf] Assuming drive cache: write through
[ 3456.457754] sdf: sdf1
[ 3456.457974] sd 11:0:0:0: [sdf] Attached SCSI disk

The partitions on those drives look like follows:

/dev/sdd1 2048 3907024895 3907022848 1.8T fd Linux raid autodetect
/dev/sde1 2048 3907024895 3907022848 1.8T fd Linux raid autodetect
/dev/sdf1 2048 3907024895 3907022848 1.8T fd Linux raid autodetect

Mounting said drives as a mirrored drive om /dev/md127:

mdadm --assemble /dev/md127 /dev/sdd1 /dev/sde1 /dev/sdf1
mdadm: /dev/md127 has been started with 3 drives.

To check the status:

mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Wed Mar  6 22:16:05 2013
        Raid Level : raid1
        Array Size : 1953380160 (1862.89 GiB 2000.26 GB)
     Used Dev Size : 1953380160 (1862.89 GiB 2000.26 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

       Update Time : Sat Mar 19 07:07:49 2022
             State : clean 
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : micemouse:0
              UUID : ed4531c4:59c132b2:a6bfc3d1:6da3b928
            Events : 6587

    Number   Major   Minor   RaidDevice State
       4       8       65        0      active sync   /dev/sde1
       5       8       81        1      active sync   /dev/sdf1
       3       8       49        2      active sync   /dev/sdd1

It's also possible to use:

mdadm --assemble --scan

Adding a drive

Previously I had only two drives, but I added one as follows:

# mdadm --grow /dev/md127 --add /dev/sdd1 --raid-devices=3
mdadm: added /dev/sdd1
raid_disks for /dev/md127 set to 3
Every 2.0s: cat /proc/mdstat                  sherlock: Sat Nov 24 15:28:29 2018

Personalities : [raid1]
md127 : active raid1 sdd1[3] sdf1[2] sde1[1]
      1953380160 blocks super 1.2 [3/2] [UU_]
      [>....................]  recovery =  0.0% (1446912/1953380160) finish=957.
9min speed=33959K/sec

unused devices: <none>

That took some time.

Thursday, 29 December 2022

Installing GraalVM on Linux using Alternatives

Was looking for how to install GraalVM on Linux using the Alternatives system.

I've already looked at sdk-man on my MacBook PRO, which works fine.

I found the references below.

References

[1] GraalVM - Installation on Linux Platforms
https://www.graalvm.org/22.3/docs/getting-started/linux/
[2] How to Install GraalVM Community Edition on Linux
https://gist.github.com/ricardozanini/fa65e485251913e1467837b1c5a8ed28

Thursday, 27 January 2022

RAID Mirrorring with MDADM

So I posted about my RAID solution plenty of times, but I have some additional information.

So here're my current harddrives:

# lsscsi
[0:0:0:0]    disk    ATA      Samsung SSD 860  1B6Q  /dev/sda 
[4:0:0:0]    disk    ATA      WDC WD4003FZEX-0 1A01  /dev/sdb 
[5:0:0:0]    disk    ATA      WDC WD5000AAKS-0 3B01  /dev/sdc 
[5:0:1:0]    disk    ATA      ST2000DM001-1CH1 CC29  /dev/sdd 
[10:0:0:0]   disk    WD       Ext HDD 1021     2021  /dev/sde 
[11:0:0:0]   disk    WD       Ext HDD 1021     2021  /dev/sdf 

Three harddrives of which I use for my RAID 1. One internal SATA drive and two external USB drives.

/dev/sdd - harddrive 2 TB
./dev/sdd1        2048 3907024895 3907022848  1.8T fd Linux raid autodetect

/dev/sde - external WD harddrive 2 TB
/dev/sde1        2048 3907024895 3907022848  1.8T fd Linux raid autodetect

/dev/sdf - external WD harddrive 2 TB
/dev/sdf1        2048 3907024895 3907022848  1.8T fd Linux raid autodetect

And then I had a problem.

# mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Wed Mar  6 22:16:05 2013
        Raid Level : raid1
        Array Size : 1953380160 (1862.89 GiB 2000.26 GB)
     Used Dev Size : 1953380160 (1862.89 GiB 2000.26 GB)
      Raid Devices : 3
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Sat Nov 27 20:57:13 2021
             State : clean, degraded 
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : micemouse:0
              UUID : ed4531c4:59c132b2:a6bfc3d1:6da3b928
            Events : 6205

    Number   Major   Minor   RaidDevice State
       2       8       65        0      active sync   /dev/sde1
       -       0        0        1      removed
       3       8       49        2      active sync   /dev/sdd1

For some reason, we're missing /dev/sdf with partition /dev/sdf1.

I am fairly concerned here.

So I reattached the device, and then had to rebuild, etc, etc and it took a few days.

mdadm /dev/md127 -a /dev/sdf1

A better solution would have been to prevent the automounting of raid arrays, until I've attached said external drives.

They are not attached by default.

Turning off auto raid detection

The reason I wanted to turn off automatic raid detection, is because I have two disks that can be attached via USB that are part of the RAID array.

When they are not attached, and the raid is autodetected, this time it started up with one device removed.

And it takes a dickens of a time to reattach the devices to the raid.

Sometimes the configuration is simply not there (autodetects the superblock on drives, and works it out from there).

If it is there, it should be either in:

/etc/mdadm/mdadm.conf
for some Linux, for example Ubuntu.
/etc/mdadm.conf
for some Linux, for example Fedora.

Simply adding the following should be enough:

echo "AUTO -all" >> /etc/mdadm.conf

A better way would probably be to have stopped the raid, and re-assemble it again.

Will try this next time if it happens.

Stopping an array

sudo umount /mnt/md127
sudo mdadm --stop --scan

Starting an Array

For simple setups:

sudo mdadm --assemble --scan

For specific setups:

sudo mdadm --assemble /dev/md127 /dev/sdd /dev/sde /dev/sdf

For machine/harddrive independent setup (he gathers the appropriate components itself):

mdadm --scan --assemble --uuid=a26bf396:31389f83:0df1722d:f404fe4c

And then you can do:

sudo mount /dev/md127 /mnt/raid

References

DigitalOcean - How To Manage RAID Arrays with mdadm on Ubuntu 16.04
https://www.digitalocean.com/community/tutorials/how-to-manage-raid-arrays-with-mdadm-on-ubuntu-16-04
Re: turn off auto assembly
https://www.spinics.net/lists/raid/msg30997.html

Thursday, 14 October 2021

My Printer's Gone!

I was missing my printer. Turns out I could just add it in the Printer Settings in Linux.

I don't know why it was suddenly missing, though.

Perhaps it happened during an upgrade of Fedora Core.

I know I installed all the required drivers some time ago.

Oh well. I thought it was something serious. It's nice to be disappointed.

References

MyBlog - Installing Printer Driver in Fedora for Epson ET-3750
https://randomthoughtsonjavaprogramming.blogspot.com/2021/03/installing-printer-driver-in-fedora-for.html

Thursday, 4 March 2021

Installing Printer Driver in Fedora for Epson ET-3750

Just some information in case I need to reinstall the drivers sometime in the future.

The following is in my log:

[ 650.114340] usb 2-1.8: new high-speed USB device number 5 using ehci-pci
[ 650.196888] usb 2-1.8: New USB device found, idVendor=04b8, idProduct=1130, bcdDevice= 1.00
[ 650.196896] usb 2-1.8: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 650.196900] usb 2-1.8: Product: ET-3750 Series
[ 650.196903] usb 2-1.8: Manufacturer: EPSON
[ 650.196907] usb 2-1.8: SerialNumber: 583445553036353185
[ 650.245866] usblp 2-1.8:1.1: usblp0: USB Bidirectional printer dev 5 if 1 alt 0 proto 2 vid 0x04B8 pid 0x1130
[ 650.245915] usbcore: registered new interface driver usblp

I found some information at [1].

[root@mrbear Downloads]# dnf install lsb

Lsb is required, or it starts complaining when trying to install the rpm packages.

rpm -i epson-inkjet-printer-escpr2-1.1.25-1lsb3.2.x86_64.rpm
rpm -i epson-printer-utility-1.1.1-1lsb3.2.x86_64.rpm

Via LPADMIN

# lpadmin -p [PRINTER_NAME] -v [DEVICE_URI] -P [PPD_FILE] -E
# lpadmin -p et3750 -v "usb://EPSON/ET-3750%20Series?serial=583445553036353185&interface=1" -P /opt/epson-stuff-this-and-that-en.ppd.gz -E

Via Webbrowser

This works if you have attached the printer via USB cable.

You can access CUPS via a webbrowser at http://localhost:631.

As administrator ask for "Add printer".

The list shows EPSON ET-3750 Series (EPSON ET-3750 Series).

Name: EPSON_ET-3750_Series
Description: EPSON ET-3750 Series
Location:
Connection: usb://EPSON/ET-3750%20Series?serial=583445553036353185&interface=1
Sharing: Do Not Share This Printer
Make: Epson
Model: (lsb/usr/epson-inkjet-printer-escpr2/Epson/Epson-ET-3750_Series-epson-inkjet-printer-escpr2.ppd.gz)
Epson ET-3750 Series - epson-inkjet-printer-escpr2 1.1.25-1lsb3.2 (Seiko Epson Corporation LSB 3.2) (en, nl, fr, de, it, ja, pt, es, ko, ru, zh_TW, zh_CN)
Or Provide a PPD File:

References

[1] Epson - Epson ET-3750
https://epson.com/Support/Printers/All-In-Ones/ET-Series/Epson-ET-3750/s/SPT_C11CG20201?review-filter=Linux

Thursday, 23 April 2020

Keyboard problems in Linux with IntelliJ

Numpad arrow keys

So the numpad arrow keys do not work in my IntelliJ editor under Fedora Core 31.

I read about similar issues on the Internet.

One of the solutions that worked for me, was to change the file /usr/share/X11/xkb/symbols/keypad.

First off copying the original file seems like a good idea, something like keypad.original.

Then replace all occurrences of KP_Up, KP_Down, KP_Left, KP_Right with Up, Down, Left, Right respectively.

Either rebooting or executing a setxkbmap should do the trick.

Ctrl-Alt-Left and Ctrl-Alt-Right

I use these to go to the next or previous call in Intellij. Linux just wants to switch Workspaces, when I do that.

The shortcuts could not be found using the standard Settings tool.

But I found a solution at [2].

gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-up []
gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-down []
gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-left []
gsettings set org.gnome.desktop.wm.keybindings switch-to-workspace-right []

With a reset if I decide to need it like:

gsettings reset org.gnome.desktop.wm.keybindings switch-to-workspace-up
gsettings reset org.gnome.desktop.wm.keybindings switch-to-workspace-down
gsettings reset org.gnome.desktop.wm.keybindings switch-to-workspace-left
gsettings reset org.gnome.desktop.wm.keybindings switch-to-workspace-right

References

Medium.com - A simple, humble but comprehensive guide to XKB for linux
https://medium.com/@damko/a-simple-humble-but-comprehensive-guide-to-xkb-for-linux-6f1ad5e13450
[2] Ubuntu - How to disable the keyboard shortcut to switch between workspaces
https://askubuntu.com/questions/744214/how-to-disable-the-keyboard-shortcut-to-switch-between-workspaces

Thursday, 19 December 2019

Java Alternatives in Fedora Core - Followup

This is a followup of the blogpost Java Alternatives in Fedora Core.

The instructions in the blogpost also hold for Java 13.

[root@localhost ~]# dnf install java-13-openjdk
[root@localhost ~]# dnf install java-13-openjdk-devel

Unfortunately, there was a snag with Maven, so I thought I'd document it here.

Maven

Maven stubbornly remained firmly pointing to java 8, whatever alternative I picked in my Fedora Core.

[mrbear@localhost ~]$ mvn -version
Apache Maven 3.5.4 (Red Hat 3.5.4-5)
Maven home: /usr/share/maven
Java version: 1.8.0_232, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.fc30.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.3.11-200.fc30.x86_64", arch: "amd64", family: "unix"

Well, trying to compile a java application with version 13 in the pom.xml, is NOT going to work when your Maven is using openjdk 8.

Really no surprise there.

You get:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project mrbearapp: Fatal error compiling: invalid target release: 13 -> [Help 1]

It turns out I really need to set JAVA_HOME?

Editing .bashrc to add the following:

export JAVA_HOME=/usr/lib/jvm/java-13

Now I get:

[mrbear@localhost ~]$ mvn -version
Apache Maven 3.5.4 (Red Hat 3.5.4-5)
Maven home: /usr/share/maven
Java version: 13.0.1, vendor: N/A, runtime: /usr/lib/jvm/java-13-openjdk-13.0.1.9-2.rolling.fc30.x86_64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.3.11-200.fc30.x86_64", arch: "amd64", family: "unix"

All is as it should be, and we are merrily on our way.

Thursday, 3 October 2019

Automating Importing Let's Encrypt Certificates into Payara/Glassfish

Well, I already made a blogpost on letsencrypt certificates in payara1.

In this blogpost I mentioned a python script that automatically renews certificates for payara2. Unfortunately, it's not quite a perfect fit for me, as I do not run payara on port 80, which the script requires.

My setup has an nginx running on port 80, that basically forwards all requests to the https version of the website.

So I could simply turn off nginx, re-certify and turn nginx back on.

This simple bash script is as follows:

Running it will give you:

$ source renew_letsencrypt_certificates.sh
---- Starting.
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/www.mrbear.org.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for www.mrbear.org
Waiting for verification...
Cleaning up challenges

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/www.mrbear.org/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/www.mrbear.org/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
---- Adding certificates...
/home/payara/payara5/glassfish/domains/production/config/keystore.jks
Private key with alias [s1as] added to keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks.
Command add-pkcs8 executed successfully.
---- Disabling http listener...
server.network-config.network-listeners.network-listener.http-listener-2.enabled=false
Command set executed successfully.
---- Enabling http listener...
server.network-config.network-listeners.network-listener.http-listener-2.enabled=true
Command set executed successfully.
---- Done.

And your certificate is renewed. I am most pleased that this is soo easy in Payara/Glassfish.

In order to properly automate this, the master password must be stored somehow, and it's possible to do this via the --passwordfile=passwordfile.txt commandline parameter.

I found a good blogpost on how to do this at .Lost in Coding3.

And also the blogpost on [4] to get me started on how to do this.

Verifying that the keystore contains the appropriate stuff, can be done as follows:

# keytool -list -keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks
Enter keystore password:
Keystore type: jks
Keystore provider: SUN

Your keystore contains 2 entries

glassfish-instance, Jan 22, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): WZ:B6:4N:8V:AT:YP:QC:9N:VT:HA:WI:NQ:B[:V8:0W:YT:B8:AW:YT:VQ
s1as, Sep 25, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): N8:5Y:AN:SV:N8:9W:4T:Y7:9W:V9:W4:VT:8Y:WV:4N:W4:VT:FB:8S:4E

To get more information, try:

keytool -list -v -keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks

Or for a specific alias:

keytool -list -v -keystore /home/payara/payara5/glassfish/domains/production/config/keystore.jks -alias s1as

References

[1] Enabling SSL in Payara with certificate from Let's Encrypt.
https://randomthoughtsonjavaprogramming.blogspot.com/2019/01/enabling-ssl-in-payara-with-certificate.html
[2] github - Let's encrypt script to retrieve and upload keys/certificates to Payara
https://gist.github.com/ratcashdev/1b09877d37e02ef5170bf9e60c377f34
[3] .Lost in Coding - Configure Payara Server passwords in Docker
https://ondro.inginea.eu/index.php/configure-passwords-in-payara-server-and-glassfish/
[4] Payara Blog - TLS certificates with Payara Server and Let's Encrypt
https://blog.payara.fish/configuring-ssl/tls-certificates-with-payara-server-and-lets-encrypt

Thursday, 6 June 2019

Converting DVD to MP4 using Linux

I recently wished to convert some of my childrens DVDs to MP4, so I could put them on their individual tablets.

I've dabbled with this in the past, on Linux, and it has always been quite a chore to do.

Everything from obscure commandline commands and GUIs that provided a lot of options that were basically meaningless to me (without proper research).

Of course, this caused a lot of grief. Sound not synchronised to the video, or the entirely wrong soundtrack, or converted only the trailers, etc.

But recently I found Handbrake1, and I was much impressed. A nice gui that only takes a little while to learn, and only for special cases (say for example if you do not wish to have the default English soundtrack).

It has sensible defaults.

The only issue was the installation, as it requires, of course, libcss and all that.

References

[1] Handbrake
https://handbrake.fr/

Thursday, 7 March 2019

Adventures in Shell Scripts

A very simple and short blogpost today, sorries.

A colleague decided to automated some simple task by means of a shell script.

This script caused the following error:

rm: cannot remove 'tmp/*': No such file or directory

I quite quickly (after adding -f to defeat the are-you-sure-prompt) found out that double quotes indicated that it should take stuff literally. In this case the script was trying to actually delete the file tmp/*, and not finding it.

The shell did not expand the * properly when encased in double quotes.

A simple example for you to try if this is the case in your environment would be:

echo *

Instead of

echo "*"

Thursday, 28 February 2019

Installing MariaDB 10 on CentOS 7

I have always used the stock install of MariaDB that came with CentOS 7, but I noticed recently that it is still only on 5.5 (at the time of writing, 5.5.60).

I never had any issues with it, until I tried to upgrade my Liferay CMS system. Apparently there are definite changes that are not incorporated into the 5.5.

I wished to moved to 10.3, which is the latest stable branch, and I can apparently do that (as indicated on the mariadb website1) by creating a MariaDB repository for DNF and then installing as usual.

Of course, first uninstall the 5.5 version.

References

[1] MariaDB - Installing MariaDB 10 on CentOS 7 / RHEL 7
https://mariadb.com/resources/blog/installing-mariadb-10-on-centos-7-rhel-7/
MariaDB - Moving to MariaDB Backup
https://mariadb.com/resources/blog/moving-to-mariadb-backup/

Thursday, 20 December 2018

Harddisks

I'm getting more and more harddisks in my computer nowadays. Some are really old, some I've managed to appropriate from family members, and of course when I upgraded to SSD, one more harddisk was added.

It's hard to keep them all straight, so I'm writing this stuff down.

For historical purposes, I guess.

It will be split up into "The New" and "The Old".

"The Old" being the original partition information at the time of acquirement.

The New

ATA Samsung SSD 860500 GBsda
The Operating System and everything essential.
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 1026047 1024000 500M 7 HPFS/NTFS/exFAT
/dev/sda2 1026048 771002103 769976056 367.2G 7 HPFS/NTFS/exFAT
/dev/sda3 771002368 771969023 966656 472M 27 Hidden NTFS WinRE
/dev/sda4 771971072 976773119 204802048 97.7G 83 Linux

ATA WDC WD4003FZEX-04 TBsdb
Very important data regarding the /home directories.
DeviceStartEndSectorsSizeType
/dev/sdb12048409520481MBIOS boot
/dev/sdb2409610280951024000500MLinux filesystem
/dev/sdb31028096527343750052724094052.5TLinux filesystem
/dev/sdb45273438208742092185521474836481TLinux filesystem

ATA WDC WD5000AAKS-0500 GBsdc
Still empty for now.
DeviceBootStartEndSectorsSizeIdType
/dev/sdc12048976773167976771120465.8G83Linux

ATA ST2000DM001-1CH12 TBsdd
An old harddrive of my mother-in-law, part of a raid now.
DeviceBootStartEndSectorsSizeIdType
/dev/sdd1 2048 3907024895 3907022848 1.8T fd Linux raid autodetect

The Old

ATA Samsung SSD 860500 GBsda
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 1026047 1024000 500M 7 HPFS/NTFS/exFAT
/dev/sda2 1026048 771002103 769976056 367.2G 7 HPFS/NTFS/exFAT
/dev/sda3 771002368 771969023 966656 472M 27 Hidden NTFS WinRE
/dev/sda4 771971072 976773119 204802048 97.7G 83 Linux

ATA WDC WD4003FZEX-04 TBsdb
DeviceStartEndSectorsSizeType
/dev/sdb12048409520481MBIOS boot
/dev/sdb2409610280951024000500MLinux filesystem
/dev/sdb31028096527343750052724094052.5TLinux filesystem
/dev/sdb45273438208742092185521474836481TLinux filesystem
/dev/sdb574209218567630637055209715200100GLinux filesystem
/dev/sdb67630637056763804467174076163.5GLinux swap

ATA WDC WD5000AAKS-0500 GBsdc
DeviceBootStartEndSectorsSizeIdType
/dev/sdc1*204820290317820290113196.8G7HPFS/NTFS/exFAT
/dev/sdc2202903552203876351972800475M27Hidden NTFS WinRE
/dev/sdc3203878400204799999921600450M27Hidden NTFS WinRE
/dev/sdc4204802048414517247209715200100G83Linux

ATA ST2000DM001-1CH12 TBsdd
DeviceBootStartEndSectorsSizeIdType
/dev/sdd1*204810240204710240000048.8G7HPFS/NTFS/exFAT
/dev/sdd21024020482000568319765478446.6G83Linux
/dev/sdd320005683220535418871853485056883.8Gfd Linux raid autodetect
/dev/sdd4205354188839070269431853485056883.8Gfd Linux raid autodetect

References

Binary Tides - 9 commands to check hard disk partitions and disk space on Linux
https://www.binarytides.com/linux-command-check-disk-partitions/

Thursday, 13 December 2018

Making your labtop battery last longer in Linux

So I went to Devoxx in Belgium, and I took my labtop along. Of course, it's in a Cinema, so there's no electrical sockets in the rooms.

I was looking to prolong the use of my labtop, so I was looking for extend the battery life.

There were several good articles about it (see below in the references.)

I choose to install the Xfce Desktop Environment, and I was pleasantly surprised my labtop managed to hold together for a lot longer.

Fedora has some notes on how to install it, it's basically running the package manager properly, and it's done.

It can be selected from the dropdown menu on the login screen.

I shall put the command line down here, so I do not forget:

sudo dnf install tlp tlp-rdw
sudo systemctl enable tlp

References

FOSS Post - 7 Tips to Reduce Battery Usage on Linux
https://fosspost.org/tutorials/7-tips-to-reduce-battery-usage-on-linux
Xfce Desktop Environment
https://www.xfce.org/
LXDE Desktop Environment for All
https://lxde.org/
Openbox - minimalistic window manager
http://openbox.org/wiki/Main_Page
Xfce In Fedora
https://fedoraproject.org/wiki/Xfce

Thursday, 29 November 2018

Java Alternatives in Fedora Core

Installing Java 11

[root@localhost ~]# dnf install java-11-openjdk
[root@localhost ~]# dnf install java-11-openjdk-devel
Last metadata expiration check: 3:26:53 ago on Mon 12 Nov 2018 09:47:59 AM CET.
Dependencies resolved.
================================================================================
 Package                   Arch       Version                 Repository   Size
================================================================================
Installing:
 java-11-openjdk-devel     x86_64     1:11.0.1.13-1.fc28      updates     3.4 M

Transaction Summary
================================================================================
Install  1 Package

Total download size: 3.4 M
Installed size: 5.1 M
Is this ok [y/N]: y
Downloading Packages:
java-11-openjdk-devel-11.0.1.13-1.fc28.x86_64.rpm                                                                                                                                                75 kB/s | 3.4 MB     00:45
----------------------------------------------------------------------------------
Total                                                     72 kB/s | 3.4 MB     00:47
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing                                       1/1
  Installing       : java-11-openjdk-devel-1:11.0.1.13-1.fc28.x86_64                                        1/1 
  Running scriptlet: java-11-openjdk-devel-1:11.0.1.13-1.fc28.x86_64                                        1/1 
                                               1/1 
  Verifying        : java-11-openjdk-devel-1:11.0.1.13-1.fc28.x86_64                                                         1/1 

Installed:
  java-11-openjdk-devel.x86_64 1:11.0.1.13-1.fc28                                                               

Complete!

Switching to another Java version

[mrbear@localhost ~]$ sudo alternatives --config java
[mrbear@localhost ~]$ sudo alternatives --config javac
There are 2 programs which provide 'javac'.

  Selection    Command
-----------------------------------------------
*  1           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181.b15-6.fc28.x86_64/bin/javac)
 + 2           java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.1.13-1.fc28.x86_64/bin/javac)

Enter to keep the current selection[+], or type selection number:

Of course my IntelliJ automatically detects the different Java versions installed on the system, when I tell it to add a new version of Java.

Nice.

References

Using alternative utils with JRE & JDK
https://robbinespu.github.io/eng/2018/03/21/Updating_java_with_alternative.html
Superhero Ninja - Easily switch between java versions using alternatives in Linux
https://superhero.ninja/2015/02/07/easily-switch-between-java-versions-using-alternatives-in-linux/

Saturday, 6 October 2018

SSH Key-Based Authentication

I'm using reference [1] almost exclusively, it is easy to follow and very well written. The reason I am writing stuff down here, is in the case the referenced article goes offline.

Generating Keys

[mrbear@localhost ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/mrbear/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Passphrases do not match. Try again.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/mrbear/.ssh/id_rsa.
Your public key has been saved in /home/mrbear/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:etv4wn8 4843n3v48nvan80avw2408a4vn588an04845 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
| ..o |
| E o= . |
| o. o |
| .. |
| ..S |
| o o. |
| =o.+. |
|. =++.. |
|o=++. |
+-----------------+
[mrbear@localhost ~]$

Copying keys to the server

[mrbear@localhost ~]$ ssh-copy-id [email protected]
The authenticity of host 'www.mysite.org (10.0.0.11)' can't be established.
ECDSA key fingerprint is SHA256:B&B(&BSf7bFSfb7fF&SFb7SF&tFibFuSOWWFuFBUf76.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

[mrbear@localhost ~]$

Logging in

[mrbear@localhost ~]$ ssh [email protected]
Last login: Mon Sep 24 21:01:12 2018

Add to sudoers file

Found how to do this at [2].

[root@mysite ~]# usermod -aG wheel mrbear

Disabling Password Authentication

Edit the file /etc/ssh/sshd_config.

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication no
# systemctl restart sshd.service

Now what you get when you try is:

$ ssh -l root www.mysite.org [email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

Thank the heavens I no longer have to worry about messages as the one below:

root@mysite's password:
Last failed login: Sat Oct 6 10:12:48 CEST 2018 from 10.0.0.1 on ssh:notty
There were 56310 failed login attempts since the last successful login.
Last login: Tue Oct 2 23:23:41 2018
[root@mysite ~]#

I originally had installed fail2ban, as a way to protect against a password-based attack, but even the man page of fail2ban3 indicates that the above way is pretty much better.

References

[1] Digital Ocean - How To Configure SSH Key-Based Authentication on a Linux Server
https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-linux-server
[2] Top 20 OpenSSH Server Best Security Practices
https://www.cyberciti.biz/tips/linux-unix-bsd-openssh-server-best-practices.html
[3] Fail2ban
https://www.fail2ban.org/wiki/index.php/Main_Page
How to disable ssh password login on Linux to increase security
https://www.cyberciti.biz/faq/how-to-disable-ssh-password-login-on-linux/

Sunday, 8 October 2017

Automatic updates with dnf-automatic

I have installed the package dnf-automatic, to allow for automatic updates to be installed.

dnf -y install dnf-automatic

There's only one configuration file available in /etc/dnf/automatic.conf.

I have set it to:

apply_updates = yes
download_updates = yes
upgrade_type = security

It shows that the timer is active:

[root@ ~]#systemctl status dnf-automatic.timer
● dnf-automatic.timer - dnf-automatic timer
Loaded: loaded (/usr/lib/systemd/system/dnf-automatic.timer; enabled; vendor preset: disabled)
Active: active (waiting) since Tue 2017-09-19 07:27:51 CEST; 2 weeks 4 days ago

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
[root@ ~]# systemctl list-timers *dnf-automatic* --all
NEXT                          LEFT         LAST                          PASSED  UNIT                ACTIVATES
Sun 2017-10-08 08:28:05 CEST  5h 8min left Sat 2017-10-07 08:28:05 CEST  18h ago dnf-automatic.timer dnf-automatic.service

1 timers listed.

References

major.io - Automatic package updates with dnf
https://major.io/2015/05/11/automatic-package-updates-with-dnf/
Linux Audit - Automatic Security Updates with DNF
https://linux-audit.com/automatic-security-updates-with-dnf/

Thursday, 10 August 2017

Excluding packages in DNF

In the file /etc/dnf/dnf.conf, you can enter which packages need to be excluded.

I like to exclude java packages, until I am ready for them (as they require a reboot of my application server):
[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=true
exclude=java*
Or for instance:
[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=true
exclude=kernel* java*
When I am ready to install all packages, regardless of excludes, I can use the commandline below.
$ dnf -y update --disableexcludes=all

References

Fedora 24: Exclude package from update
https://www.hiroom2.com/2016/07/07/fedora-24-exclude-package-from-update/
SysTutorials - Making dnf/yum Not Update Certain Packages
https://www.systutorials.com/1661/making-dnf-yum-not-update-certain-packages/

Thursday, 20 July 2017

mount: unknown filesystem type 'exfat'

When attempting to mount a USB drive, I encountered the following error regarding the filesystem exFAT1:
mount: unknown filesystem type 'exfat'
I had to install fuse-exfat as detailed according to [2].
# yum install fuse-exfat
Redirecting to '/usr/bin/dnf install fuse-exfat' (see 'man yum2dnf')

Last metadata expiration check: 0:00:02 ago on Fri May 12 07:06:52 2017.
Dependencies resolved.
================================================================================
Package        Arch       Version             Repository                  Size
================================================================================
Installing:
fuse-exfat     x86_64     1.2.5-1.fc25        rpmfusion-free-updates      40 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 40 k
Installed size: 71 k
Is this ok [y/N]: Y
Downloading Packages:
fuse-exfat-1.2.5-1.fc25.x86_64.rpm              335 kB/s |  40 kB     00:00    
--------------------------------------------------------------------------------
Total                                           118 kB/s |  40 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Installing  : fuse-exfat-1.2.5-1.fc25.x86_64                              1/1
Mounting after that worked flawlessly.
# mount /dev/sdb1 mydrive
FUSE exfat 1.2.5

References

[1] Wikipedia - exFAT
https://en.wikipedia.org/wiki/ExFAT
[2] Mounting EXFAT formatted pendrives in fedora linux
https://coderwall.com/p/nvwgea/mounting-exfat-formatted-pendrives-in-fedora-linux