Description

Ive been running an SRX210B for a while and decided get a hardware upgrade. A few things pushed me to get a better model. SNMP monitoring the 210 with zabbix was killing the CPU. To not overload the host i had to selectively monitor GE interfaces and the 1 FE interface configured for outbound traffic. The RAM was always pushing 83% used, with Control plane memory over 90%.

The SRX210 has 2xGE or 1000Mbit interfaces and 6xFE or 100Mbit interfaces. In the default junos config a the first (GE) interface is configured as the untrust interface, as my broadband is less than 100Mbit this wastes a fast port so i configured the last FE/7 interface as untrust, i then seperated my server and wifi on the 2 GE interfaces.

Those FE interfaces bothered me and i wanted all Gbit. I really dont understand why the 210 shipped with 2xGE 1000bit and 6xFE 100Mbit ports, surely if they wanted to limit capacity or throughput to distinguish the budget branch model they would do that on the firewall throughput alone. Very puzzling, very strange, but otherwise cheap and easy to acquire SRX branch security gateway to learn Junos. You can commonly find the 210 for £20 on ebay.

So to be clear, saturating CPU, high memory usage, having slower 100Mbit ports, having to switch the untrust interfece to fe-7 and wanting to run a more upto date junos also pushed me towards getting the SRX220H2.

Product EOL

https://support.juniper.net/support/eol/product/srx_series/

Product/SKU(s) EOL Announced Last Order End of Support
SRX220H2 05/30/2018 11/30/2018 11/30/2023

Upgrade

You can find the latest supported Junos release for the SRX220H2 here srx220 downloads. Thats 12.3X48-D105 at this time and the checksums below.

MD5 : 7fc14d3a965609bafaf9ca730c00a959

SHA1 : e0b4c51a930537f24ae22425e34742c45579b3cf

SHA256 : d08142898d4bc2d4cc12b608082c461d8035728dd12ea1f6bd9a3b92668ce491

SHA512 : 096cb5aa739339a05cb00ac0e20153077026b8360da9034adf636f088cc43aff6bb4c4db4385ecb381770d4842d2fbf2c1f39a66c58596b7645420ac63b2efed

Downloading 12.3X48-D105

Luckily some of the Junos releases are now on the internet archive including 12.3X48-D105! I went ahead and grabbed the release from the following URL and put it on a USB drive.

https://archive.org/download/junos-srxsme

Before doing anything i verified this is the true package with the sha512 checksum.

% sha512sum junos-srxsme-12.3X48-D105.4-domestic.tgz
096cb5aa739339a05cb00ac0e20153077026b8360da9034adf636f088cc43aff6bb4c4db4385ecb381770d4842d2fbf2c1f39a66c58596b7645420ac63b2efed  junos-srxsme-12.3X48-D105.4-domestic.tgz

You can copy the release over the network to /var/tmp on the SRX but you wont be able to see the full upgrade remotely (reboots) to do that you need console cable. As ive covered this before ill just link he 210B recovery and upgrade post here /blog/2020/02/18/srx210-recovery-upgrade.html

Proceedure for upgrade

Start by getting the current version and chassis status

UNAUTHORIZED USE OF THIS ROUTER
        IS STRICTLY PROHIBITED!


core-220 (ttyu0)

login: root
Password:

--- JUNOS 12.1X46-D65.4 built 2016-12-30 01:34:30 UTC
root@core-220% cli
root@core-220> show version 
Hostname: core-220
Model: srx220h2
JUNOS Software Release [12.1X46-D65.4]

root@core-220> show chassis alarms 
No alarms currently active

root@core-220> exit

Find the USB device and upgrade package

root@core-220% ls -l /dev/da*
crw-r-----  1 root  operator    0,  88 Jul 24 16:22 /dev/da0
crw-r-----  1 root  operator    0,  89 Jul 24 16:22 /dev/da0s1
root@core-220% mount_msdosfs /dev/das0s1 /mnt
root@core-220% ls -l /mnt
total 652688
-r-xr-xr-x  1 root  wheel  149328506 Feb  7  2020 junos-srxsme-12.1X46-D65.4-domestic.tgz
-r-xr-xr-x  1 root  wheel  184715990 Jul 29 10:21 junos-srxsme-12.3X48-D105.4-domestic.tgz
root@core-220% cli

Request upgrade to 12.3X48-D105.4

root@core-220> request system software add /mnt/junos-srxsme-12.3X48-D105.4-domestic.tgz 
NOTICE: Validating configuration against junos-srxsme-12.3X48-D105.4-domestic.tgz.
NOTICE: Use the 'no-validate' option to skip this if desired.
Checking compatibility with configuration
Initializing...
Verified manifest signed by PackageProductionEc_2016 method ECDSA
veriexec: cannot bless /packages/junos-12.1X46-D65.4-domestic: Authentication error
Verified junos-12.1X46-D65.4-domestic signed by PackageProductionEc_2016 method ECDSA
Using /mnt/junos-srxsme-12.3X48-D105.4-domestic.tgz
Checking junos requirements on /
Available space: 634340 require: 213826
Verified junos-boot-srxsme-12.3X48-D105.4.tgz signed by PackageProductionECP256_2020 method ECDSA
Verified junos-srxsme-12.3X48-D105.4-domestic signed by PackageProductionECP256_2020 method ECDSA
Saving boot file package in /var/sw/pkg/junos-boot-srxsme-12.3X48-D105.4.tgz
veriexec: cannot update veriexec for /cf/var/validate/chroot/junos/usr/lib/libslax.so.3: No such file or directory
veriexec: cannot update veriexec for /cf/var/validate/chroot/junos/usr/lib/libext_bit.so.3: No such file or directory
veriexec: cannot update veriexec for /cf/var/validate/chroot/junos/usr/lib/libext_curl.so.3: No such file or directory
veriexec: cannot update veriexec for /cf/var/validate/chroot/junos/usr/lib/libext_xutil.so.3: No such file or directory
Verified manifest signed by PackageProductionECP256_2020 method ECDSA
Hardware Database regeneration succeeded
Validating against /config/juniper.conf.gz
Usage: license-check -f "<features>" -m -p -q -M -u -U -V
  -V verify if release based licenses are present
Chassis control process: <xnm:warning xmlns="http://xml.juniper.net/xnm/1.1/xnm" xmlns:xnm="http://xml.juniper.net/xnm/1.1/xnm">
Chassis control process: <source-daemon>chassisd</source-daemon>
Chassis control process: <message>realtime-ukernel-thread is disable. Please use the command request system reboot.</message>
Chassis control process: </xnm:warning>
Connectivity fault management process: rtslib: ERROR kernel does not support all messages: expected 103 got 102,a reboot or software upgrade may be required
Connectivity fault management process:
Connectivity fault management process: rtslib: WARNING version mismatch for msg bulkstats: expected 0 got 98,a reboot or software upgrade may be required
Connectivity fault management process:
Connectivity fault management process: rtslib: WARNING version mismatch for msg ddos: expected 99 got 98,a reboot or software upgrade may be required
Connectivity fault management process:
Connectivity fault management process: rtslib: WARNING version mismatch for msg notify msg: expected 98 got 0,a reboot or software upgrade may be required
Connectivity fault management process:
Connectivity fault management process: rtslib: WARNING version mismatch for msg kuack messages: expected 99 got 0,a reboot or software upgrade may be required
Connectivity fault management process:
Connectivity fault management process: rtslib: WARNING version mismatch for msg marker ifstate: expected 104 got 0,a reboot or software upgrade may be required
Connectivity fault management process:
mgd: commit complete
Validation succeeded
Formatting alternate root (/dev/ad0s1a)...
/dev/ad0s1a: 629.5MB (1289196 sectors) block size 16384, fragment size 2048
        using 4 cylinder groups of 157.38MB, 10072 blks, 20224 inodes.
super-block backups (for fsck -b #) at:
 32, 322336, 644640, 966944
Extracting /mnt/junos-srxsme-12.3X48-D105.4-domestic.tgz ...
saving package file in /var/sw/pkg ...
Installing package '/altroot/cf/packages/install-tmp/junos-12.3X48-D105.4-domestic' ...
Verified junos-boot-srxsme-12.3X48-D105.4.tgz signed by PackageProductionECP256_2020 method ECDSA
Verified junos-srxsme-12.3X48-D105.4-domestic signed by PackageProductionECP256_2020 method ECDSA
Verified junos-boot-srxsme-12.3X48-D105.4.tgz signed by PackageProductionECP256_2020 method ECDSA
Verified junos-srxsme-12.3X48-D105.4-domestic signed by PackageProductionECP256_2020 method ECDSA
Verified junos-boot-srxsme-12.3X48-D105.4.tgz signed by PackageProductionECP256_2020 method ECDSA
Verified junos-srxsme-12.3X48-D105.4-domestic signed by PackageProductionECP256_2020 method ECDSA
JUNOS 12.3X48-D105.4 will become active at next reboot
WARNING: A reboot is required to load this software correctly
WARNING:     Use the 'request system reboot' command
WARNING:         when software installation is complete
Saving state for rollback ...

root@core-220> exit

Umount the storage device and reboot

root@core-220% umount /mnt
root@core-220% cli
root@core-220> request system reboot 
Reboot the system ? [yes,no] (no) yes 

Shutdown NOW!
[pid 9827]

root@core-220>
*** FINAL System shutdown message from root@core-220 ***

System going down IMMEDIATELY                                                  

                                                                               
JWaiting (max 60 seconds) for system process `vnlru_mem' to stop...done
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 0 0 done

syncing disks... All buffers synced.
Uptime: 6d3h19m18s
Rebooting...
cpu_reset: Stopping other CPUs


U-Boot 1.1.6-JNPR-2.5 (Build time: Apr  2 2013 - 12:59:56)

Initializing memory this may take some time...
Measured DDR clock 265.58 MHz
SRX_220H2 board revision major:1, minor:4, serial #: ACLC6822
OCTEON CN5020-SCP pass 1.1, Core clock: 700 MHz, DDR clock: 265 MHz (530 Mhz data rate)
DRAM:  2048 MB
Starting Memory POST... 
Checking datalines... OK
Checking address lines... OK
Checking 512K memory for U-Boot... OK.
Running U-Boot CRC Test... OK.
Flash:  8 MB
USB:   scanning bus for devices... 3 USB Device(s) found
       scanning bus for storage devices... 1 Storage Device(s) found
Clearing DRAM........ done
BIST check passed.
Starting PCI
PCI Status: PCI 32-bit
PCI BAR 0: 0xf8000000, PCI BAR 1: Memory 0x00000000  PCI 0x00000000
Warning!!!Last reboot reason 0x0 abnormal
Boot Media: usb internal-compact-flash 
Net:   octeth0

  ide 0: Model: CF 2GB Firm: 20100924 Ser#: 2014A     0000002550
            Type: Removable Disk
            Capacity: 2000.7 MB = 1.9 GB (4097520 x 512)
POST Passed
Press SPACE to abort autoboot in 1 seconds
ELF file is 32 bit
Loading .text @ 0x8f0000a0 (246560 bytes)
Loading .rodata @ 0x8f03c3c0 (14144 bytes)
Loading .reginfo @ 0x8f03fb00 (24 bytes)
Loading .rodata.str1.4 @ 0x8f03fb18 (16516 bytes)
Loading set_Xcommand_set @ 0x8f043b9c (96 bytes)
Loading .rodata.cst4 @ 0x8f043bfc (20 bytes)
Loading .data @ 0x8f044000 (5744 bytes)
Loading .data.rel.ro @ 0x8f045670 (120 bytes)
Loading .data.rel @ 0x8f0456e8 (136 bytes)
Clearing .bss @ 0x8f045770 (11600 bytes)
## Starting application at 0x8f0000a0 ...
Consoles: U-Boot console  
Found compatible API, ver. 2.5

FreeBSD/MIPS U-Boot bootstrap loader, Revision 2.5
([email protected], Tue Apr  2 12:36:46 PDT 2013)
Memory: 2048MB
[0]Booting from internal-compact-flash slice 1
Un-Protected 1 sectors
writing to flash...
Protected 1 sectors
Loading /boot/defaults/loader.conf 
/kernel data=0xba1cf8+0x13c97c syms=[0x4+0x91da0+0x4+0xd622c]


Hit [Enter] to boot immediately, or space bar for command prompt.
Booting [/kernel] in 1 second... ^MBooting [/kernel]...               
Kernel entry at 0x801000c0 ...
init regular console
Primary ICache: Sets 64 Size 128 Asso 4
Primary DCache: Sets 1 Size 128 Asso 64
Secondary DCache: Sets 128 Size 128 Asso 8
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
kld_map_v: 0x8ff80000, kld_map_p: 0x0
Copyright (c) 1996-2020, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
JUNOS 12.3X48-D105.4 #0: 2020-07-29 00:24:05 UTC
    [email protected]:/volume/build/junos/12.3/service/12.3X48-D105.4/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
JUNOS 12.3X48-D105.4 #0: 2020-07-29 00:24:05 UTC
    [email protected]:/volume/build/junos/12.3/service/12.3X48-D105.4/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
real memory  = 2147483648 (2048MB)
avail memory = 1040236544 (992MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
Security policy loaded: Junos MAC/veriexec (mac_veriexec)
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
MAC/veriexec fingerprint module loaded: SHA256
MAC/veriexec fingerprint module loaded: SHA1
netisr_init: !debug_mpsafenet, forcing maxthreads from 2 to 1
cpu0 on motherboard
: CAVIUM's OCTEON 5020 CPU Rev. 0.1 with no FPU implemented
        L1 Cache: I size 32kb(128 line), D size 8kb(128 line), sixty four way.
        L2 Cache: Size 128kb, 8 way
obio0 on motherboard
uart0: <Octeon-16550 channel 0> on obio0
uart0: console (9600,n,8,1)
twsi0 on obio0
dwc0: <Synopsis DWC OTG Controller Driver> on obio0
usb0: <USB Bus for DWC OTG Controller> on dwc0
usb0: USB revision 2.0
uhub0: vendor 0x0000 DWC OTG root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1: vendor 0x0409 product 0x005a, class 9/0, rev 2.00/1.00, addr 2
uhub1: single transaction translator
uhub1: 3 ports with 2 removable, self powered
umass0: CBM Flash Disk, rev 2.00/1.00, addr 3
cpld0 on obio0
pcib0: <Cavium on-chip PCI bridge> on obio0
Disabling Octeon big bar support
PCI Status: PCI 32-bit: 0xc041b
pcib0: Initialized controller
pci0: <PCI bus> on pcib0
pci0: <simple comms> at device 1.0 (no driver attached)
atapci0: <SiI 0680 UDMA133 controller> port 0x8-0xb,0x10-0x17,0x18-0x1b,0x20-0x2f mem 0x8020000-0x80200ff irq 0 at device 2.0 on pci0
ata2: <ATA channel 0> on atapci0
ata3: <ATA channel 1> on atapci0
gblmem0 on obio0
octpkt0: <Octeon RGMII> on obio0
cfi0: <AMD/Fujitsu - 8MB> on obio0
Timecounter "mips" frequency 700000000 Hz quality 0
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
ad0: Device does not support APM
ad0: 2000MB <CF 2GB 20100924> at ata2-master WDMA2
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <CBM Flash Disk 5.00> Removable Direct Access SCSI-2 device 
da0: 40.000MB/s transfers
da0: 1010MB (2068480 512 byte sectors: 64H 32S/T 1010C)
Kernel thread "wkupdaemon" (pid 47) exited prematurely.
Trying to mount root from ufs:/dev/ad0s1a
MFSINIT: Initialising MFSROOT 
Process-1 beginning MFSROOT initialization...
Creating MFSROOT...
/dev/md0: 20.0MB (40956 sectors) block size 16384, fragment size 2048
        using 4 cylinder groups of 5.00MB, 320 blks, 640 inodes.
super-block backups (for fsck -b #) at:
 32, 10272, 20512, 30752
Populating MFSROOT...
Creating symlinks...
Setting up mounts...
Continuing boot from MFSROOT...
Attaching /cf/packages/junos via /dev/mdctl...
Mounted junos package on /dev/md1...
J
Automatic reboot in progress...
** /dev/ad0s1a (NO WRITE)
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
161 files, 92522 used, 224648 free (32 frags, 28077 blocks, 0.0% fragmentation)
mount reload of '/' failed: Operation not supported 

Verified jboot signed by PackageProductionECP256_2020 method ECDSA256+unknown
Verified junos signed by PackageProductionECP256_2020 method ECDSA256+unknown
Verified junos-12.3X48-D105.4-domestic signed by PackageProductionECP256_2020 method ECDSA256+unknown
Checking integrity of BSD labels:
  s1: Passed
  s2: Passed
  s3: Passed
  s4: Passed
** /dev/bo0s3e
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 24922 free (18 frags, 3113 blocks, 0.1% fragmentation)
** /dev/bo0s3f
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 148300 free (100 frags, 18525 blocks, 0.0% fragmentation)
Checking integrity of licenses:
Checking integrity of configuration:
  rescue.conf.gz: No recovery data
Loading configuration ...
Time and ticks drifted too much,                        resetting synchronization...

Non-existant dump device /dev/bo0s1b
mgd: commit complete
Setting initial options: .
Starting optional daemons:  usbd.
Doing initial network setup:.
Initial interface configuration:
additional daemons: eventd.
Non-existant dump device /dev/bo0s1b
Additional routing options:kern.module_path: /boot//kernel;/boot/modules -> /boot/modules;/modules/ifpfe_drv;/modules;
kld netpfe drv: ifpfed_dialer ipsec kld.
Doing additional network setup:.
Starting final network daemons:.
setting ldconfig path: /usr/lib /opt/lib
starting standard daemons: cron.
Initial rc.mips initialization:.
Local package initialization:.
starting local daemons:set cores for group access
.
kern.securelevel: -1 -> 1
Creating JAIL MFS partition...
JAIL MFS partition created
Boot media /dev/ad0 has dual root support
WARNING: JUNOS versions running on dual partitions are not same
** /dev/ad0s2a
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 241483 free (27 frags, 30182 blocks, 0.0% fragmentation)
Fri Jul 30 19:45:30 UTC 2021

Show the root and alt root versions

UNAUTHORIZED USE OF THIS ROUTER
        IS STRICTLY PROHIBITED!


core-220 (ttyu0)

login: root
Password:

--- JUNOS 12.3X48-D105.4 built 2020-07-29 00:24:05 UTC
root@core-220% 
root@core-220% cli
root@core-220> show version 
Hostname: core-220
Model: srx220h2
JUNOS Software Release [12.3X48-D105.4]

root@core-220> show chassis alarms 
No alarms currently active

root@core-220> show system snapshot media internal
Information for snapshot on       internal (/dev/ad0s1a) (primary)
Creation date: Jul 30 19:43:32 2021
JUNOS version on snapshot:
  junos  : 12.3X48-D105.4-domestic
Information for snapshot on       internal (/dev/ad0s2a) (backup)
Creation date: Jul 20 20:06:39 2021
JUNOS version on snapshot:
  junos  : 12.1X46-D65.4-domestic

Lets not waste time and just upgrade the alt root right now

root@core-220> request system snapshot slice alternate 
Formatting alternate root (/dev/ad0s2a)...
Copying '/dev/ad0s1a' to '/dev/ad0s2a' .. (this may take a few minutes)
The following filesystems were archived: /

And last review of status

root@core-220> show system snapshot media internal 
Information for snapshot on       internal (/dev/ad0s1a) (primary)
Creation date: Jul 30 19:50:40 2021
JUNOS version on snapshot:
  junos  : 12.3X48-D105.4-domestic
Information for snapshot on       internal (/dev/ad0s2a) (backup)
Creation date: Jul 30 19:48:26 2021
JUNOS version on snapshot:
  junos  : 12.3X48-D105.4-domestic

At this point we could also issue the following command to boot into the alternative root. Im going to skip validating this for now.

> request system reboot slice alternate media internal

First impressions

So whats the overhead of going from 12.1 to 12.3 ? memory usage went from 34 to 39% otherwise nothing noticable at this stage of the configuration.

As far as SNMP monitoring is going i have not had to disable anything from the base zabbix junos template, unlike the 210 where many discovered items were disabled or intervals were increased to bring CPU usage down from 80% to on average 20%, the 220 is at 7% without tweaking or disabling any discovered items. Looking at the hardware both devices share the same CPU just clocked at different rates shown below.

SRX220H2:

OCTEON CN5020-SCP pass 1.1, Core clock: 700 MHz, DDR clock: 265 MHz

SRX210B:

OCTEON CN5020-SCP pass 1.1, Core clock: 500 MHz, DDR clock: 266 MHz

The 210 is perfectly fine device but the official Junos release is old now. The 210 is also pretty much silent where as the 220 is more a rack device you dont want it in a living or working space, i presume the noise is down to the active cooling for hte higher clock rate.

Next ill document seperation of networks on the device in a homelab setup.