In previous writeups such as xmrig with cuda for Rocky Linux 8.5 and nVidia CUDA with the wrong video card I’ve navigated Rocky Linux and Cuda. It’s now time to see if we can get John the Ripper CUDA’s components running on a Rocky 8.7 Live Workstation USB install.
Personally, I love projects like this. I started this on 1 8GB USB stick and quickly realized that not only the space required wasn’t enough but I’d need more to do what I needed. I ended up getting 3 SanDisk 32GB Ultra USB 3.0 Flash Drives from Amazon for $16.96.
The biggest help with the Live USB install is using balenaEtcher to get the 2.1GB ISO to an 32GB USB stick. Once that’s done we can boot directly to the Live OS and start our installs.
I did have some derps with balenaEtcher failing to burn the ISO due to a failure of diskpart not returning a positive result to the clean operation. To resolve this I had to use PowerISO to clean the USB volume before windows would properly do it’s clean operation. Minor note to PowerISO is that it contains bloatware during the install and a wrongly-clicked click can give you headaches.
Live Stuff
- Booted up Rocky 8.7 Workstation Live Workstation from a USB to install Rocky 8.7 Workstation on a separate USB stick.
- Root with password, user with password
- rebooted into USB bootable
- #win
Now onto the necessities to get to our final goal
Downloads and Installs
Fresh install and first boot leaves up with a bit of free space, albeit many of the nVidia and CUDA installs are hefty we might have to do some space-shifting to make sure this all works.
[user@johncuda ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 9.5M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/mapper/rl_localhost--live-root 25G 5.4G 20G 22% /
tmpfs 3.9G 60K 3.9G 1% /tmp
/dev/sdb1 1014M 264M 751M 26% /boot
tmpfs 789M 24K 789M 1% /run/user/1000There’s a ton of typical set up for something like this, such as updating everything. On an Intel(R) Core(TM)2 Quad CPU Q9650 @ 3.00GHz system running off a USB stick this takes quite a period of time.
$ sudo yum install epel-release
$ sudo dnf config-manager --set-enabled powertools
$ sudo /usr/bin/crb enable
$ sudo time dnf install kernel-devel-$(uname -r)
...
34.76user 11.92system 1:24.75elapsed 55%CPU (0avgtext+0avgdata 191272maxresident)k
24992inputs+578976outputs (5major+1733808minor)pagefaults 0swapsInstalling CUDA Toolkit
Downloading the nVidia CUDA Tooklit for Rocky 8 was a breeze. Installing it, however, took it’s sweet time
$ time sudo rpm -i cuda-repo-rhel8-12-1-local-12.1.0_530.30.02-1.x86_64.rpm
warning: cuda-repo-rhel8-12-1-local-12.1.0_530.30.02-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 1dcc03d4: NOKEY
real 46m55.862s
user 0m48.051s
sys 0m7.276s
Ran all the commands, rebooted, and cuda was installed but it seems that nvidia-driver:latest-dkms didn’t complete. Testing nvidia-smi resulted in the following:
[user@johncuda ~]$ nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.nVidia Driver
Per nVidia CUDA with the wrong video card we get to install the latest nVidia driver. Instead of downloading the 470.103.01 (Jan 31, 2022) driver from the previous post I’m gonna get the latest 470.182.03 (Mar 30, 2023) driver for our Geforce GT630.
The installer needed specific steps, such as “Continue” and “No” before it began to build new Kernel Modules. Clicked an awesome “No” for compatibility library installation. Received an error about libglvnd, which I don’t need. Finished with the nvidia-xconfig run and things finalized without an issue. Rebooted one more time to make sure I’m in the green and it completely failed.
As a minor note it’s best to install all this stuff while X is not running. Quickly going to runlevel 4 and rerunning the driver install made it much easier and quicker things to finally work once kicking back on X.
Prior to figuring all this out there were more weird steps. Rocky 8.7 workstation comes with the nouveau driver, an open-sourced nVidia driver, and this installation required that it be disabled.
$ nvidia-smi
Tue Apr 18 11:59:12 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.182.03 Driver Version: 470.182.03 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 N/A | N/A |
| N/A 35C P8 N/A / N/A | 192MiB / 981MiB | N/A Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
John & CUDA
A quick link out to John the Ripper we get 1.9.0 jumbo downloaded.
./configure led us to needing openssl-dev and opencl-headers. Compiling was quick and easy. Checking our built executable and the necessary support:
[user@johncuda run]$ ./john --list=opencl-devices
Platform #0 name: NVIDIA CUDA, version: OpenCL 3.0 CUDA 11.4.364
Device #0 (1) name: NVIDIA GeForce GT 630
Device vendor: NVIDIA Corporation
Device type: GPU (LE)
Device version: OpenCL 3.0 CUDA
Driver version: 470.182.03 [recommended]
Native vector widths: char 1, short 1, int 1, long 1
Preferred vector width: char 1, short 1, int 1, long 1
Global Memory: 981 MB
Global Memory Cache: 32 KB
Local Memory: 48 KB (Local)
Constant Buffer size: 64 KB
Max memory alloc. size: 245 MB
Max clock (MHz): 901
Profiling timer res.: 1000 ns
Max Work Group Size: 1024
Parallel compute cores: 2
CUDA cores: 384 (2 x 192)
Speed index: 345984
Warp size: 32
Max. GPRs/work-group: 65536
Compute capability: 3.5 (sm_35)
Kernel exec. timeout: yes
NVML id: 0
PCI device topology: 01:00.0
Temperature: 38°C
Utilization: n/a
Now we’re good to go!