The problem

You’ve set up a mirror (SPAN) session on your switch and patched the destination port into a spare NIC on an OpenShift node. You want two VMs to see that traffic: say, an IDS like Suricata and a packet capture or Zeek box. One or both may also need to send traffic back out through that port.

This sounds simple, but most of the obvious ways to give a VM a physical network don’t work for mirrored traffic. Mirrored frames aren’t addressed to your VMs. They’re copies of conversations between other hosts, so their destination MACs belong to machines that may not even be on this network segment. Anything that forwards based on destination MAC will quietly drop most of them.

This post walks through the options, explains why most of them fail, and shows a working setup that uses a Linux bridge running as a hub.

My lab host, snok, is a single-node OpenShift 4.22 cluster with OpenShift Virtualization on an ASRock Rack EPYCD8 board. It has two spare onboard NICs (enp65s0f0, enp65s0f1) next to the 25G bond that carries cluster traffic. The examples use those names, so substitute your own.

The options

The winner is the last row. A Linux bridge with MAC learning turned off behaves like an old-fashioned Ethernet hub: every frame coming in the mirror port goes out to every attached VM.

Architecture

  Switch                         OpenShift node (snok-fast)
 ┌────────────┐                ┌──────────────────────────────────────────┐
 │ SPAN dest  │── patch cable ─┤ enp65s0f0 (promisc, GRO/LRO off, no IP)  │
 │   port     │                │      │                                   │
 └────────────┘                │  ┌───┴──────────────────────────┐        │
                               │  │ br-mirror (linux-bridge)      │        │
                               │  │  mac-ageing-time: 0  → hub    │        │
                               │  │  stp off, mcast snooping off  │        │
                               │  └───┬───────────────────┬───────┘        │
                               │      │ tap               │ tap            │
                               │  ┌───┴────┐         ┌────┴───┐            │
                               │  │ VM  A  │         │ VM  B  │            │
                               │  │ eth1   │         │ eth1   │            │
                               │  │ promisc│         │ promisc│            │
                               │  └────────┘         └────────┘            │
                               └──────────────────────────────────────────┘

Three pieces make this work:

1. A NodeNetworkConfigurationPolicy (NNCP) that builds br-mirror on the node with the spare NIC as its only uplink.
2. A NetworkAttachmentDefinition (NAD) that exposes that bridge to VMs using the bridge CNI plugin.
3. A second interface on each VM attached to that NAD.

Step 1: Build the hub-mode bridge on the node

The Kubernetes NMState Operator manages host networking declaratively. This policy takes over the spare NIC, turns off the offloads that would hide what’s really on the wire, and puts it in a bridge with MAC learning disabled.

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: br-mirror
spec:
  nodeSelector:
    kubernetes.io/hostname: snok-fast
  desiredState:
    interfaces:
    - name: enp65s0f0
      type: ethernet
      state: up
      mtu: 9000
      ipv4: {enabled: false}
      ipv6: {enabled: false}
      ethtool:
        feature:
          rx-gro: false        # don't coalesce frames — deliver what was on the wire
          rx-lro: false
    - name: br-mirror
      type: linux-bridge
      state: up
      mtu: 9000
      ipv4: {enabled: false}
      ipv6: {enabled: false}
      bridge:
        options:
          stp:
            enabled: false
          mac-ageing-time: 0           # never learn MACs → flood every frame (hub)
          multicast-snooping: false    # flood multicast to every port too
          group-forward-mask: 16384    # also forward LLDP (01:80:c2:00:00:0e)
        port:
        - name: enp65s0f0

What each setting does:

• mac-ageing-time: 0 is the key setting. Normally a bridge learns which MAC lives behind which port and forwards only there. With ageing at zero, learned entries expire immediately, so the bridge never has a forwarding entry and floods every unicast frame to all ports. That’s exactly how a hub behaves.
• multicast-snooping: false: with snooping on, the bridge sends multicast only to ports that joined the group. Your sensors haven’t joined anything, so turn it off and let multicast flood too.
• group-forward-mask: 16384: Linux bridges consume IEEE link-local frames (01:80:c2:00:00:0X) by default. Bit 14 (0x4000) lets LLDP through, which is handy for an IDS. The kernel won’t let you forward STP, PAUSE or LACP (bits 0–2) regardless.
• GRO/LRO off: with these on, the NIC merges consecutive TCP segments into large super-frames before the bridge sees them. Capture tools then report packets that never existed on the wire, and IDS reassembly can get confused.
• No IP on the NIC or bridge: the host doesn’t need to take part in this network. It just forwards frames.
• MTU 9000: if any mirrored source port carries jumbo frames, the copies will be jumbo too. Undersized MTUs drop them silently.

You don’t need to set promiscuous mode on the NIC. The kernel enables it automatically on any interface added to a bridge.

VLAN filtering is left off (the nmstate default when you don’t configure VLANs on the ports), so 802.1Q-tagged frames from a trunk mirror pass through with their tags intact.

Apply it and check:

oc apply -f nncp-br-mirror.yaml
oc get nncp br-mirror
oc get nnce
# On the node:
cat /sys/class/net/br-mirror/bridge/ageing_time   # → 0
ip -d link show enp65s0f0 | grep -o promiscuity.*  # → promiscuity 1

Step 2: Expose the bridge to VMs

Create a NetworkAttachmentDefinition in the namespace where your VMs live:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: mirror-tap
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/resourceName: bridge.network.kubevirt.io/br-mirror
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "name": "mirror-tap",
      "type": "bridge",
      "bridge": "br-mirror",
      "mtu": 9000,
      "macspoofchk": true,
      "ipam": {}
    }

Notes:

• The resourceName annotation makes the scheduler place VMs only on nodes where br-mirror exists. On a single-node cluster that doesn’t matter much, but it costs nothing and protects you if you add nodes later.
• ipam: {}: no addresses are handed out. Sensors listen and don’t need IPs on the capture interface.
• macspoofchk: true only affects traffic leaving the VM. It stops the VM from sending frames with any MAC except its own. Leave it on for passive sensors. Set it to false only if a VM needs to inject traffic with other source MACs (for example, an inline tool or one that forges TCP resets).

Step 3: Attach the VMs

Add a second interface to each VM. Keep the VM’s normal management network as-is.

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: sensor-a
  namespace: default
spec:
  template:
    spec:
      domain:
        devices:
          interfaces:
          - name: default
            masquerade: {}
          - name: mirror
            bridge: {}
            model: virtio
      networks:
      - name: default
        pod: {}
      - name: mirror
        multus:
          networkName: mirror-tap

Repeat for sensor-b. Every VM attached to mirror-tap gets its own tap device on br-mirror, and the hub floods every frame to all of them.

Step 4: Configure the guests

Inside each guest, bring the capture interface up with no address and in promiscuous mode:

ip link set eth1 up promisc on
ip link set eth1 mtu 9000
# Turn off offloads inside the guest as well, for accurate captures
ethtool -K eth1 gro off lro off 2>/dev/null || true

Then point your tools at it:

tcpdump -i eth1 -nn -c 20
zeek -i eth1 local
suricata -i eth1

On a systemd-networkd or NetworkManager guest, make this persistent with a profile that has no IP configuration and promiscuous mode enabled, so it comes back after reboots.

Gotchas worth knowing before you start

1. Can the VMs “connect” through the mirror port?

Often not. Many switches ignore incoming traffic on a SPAN destination port by default. It’s a one-way copy. If your VMs need to send traffic (active scanning, TCP resets, or simply having an IP on that segment), you have two choices:

• Enable ingress on the mirror session. On Cisco IOS that’s monitor session 1 destination interface Gi1/0/10 ingress vlan 20, and other vendors have similar settings.
• Use a second, normal access port (on snok that’s the other spare NIC, enp65s0f1) for anything the VMs need to send, and keep the mirror port receive-only. This is usually the cleaner design.

2. The VMs can see each other

A hub is a hub. Anything sensor A transmits on its mirror interface goes to the physical port and to sensor B. For passive sensors that never send, it doesn’t matter. For anything active, keep it in mind.

3. Oversubscription happens on the switch, not the host

If you mirror several busy 10G or 25G ports into a 1G destination, the switch drops the excess before it ever reaches the NIC, and nothing on the host will tell you. Check the destination port speed (ethtool enp65s0f0) and the switch’s drop counters on the mirror session. If you need more, use a faster destination port or mirror less.

4. nmstate must be able to verify the change

After applying a policy, nmstate checks that the node still works. It pings the default gateway(s), checks DNS and contacts the API server. If any of those fail, it rolls the change back, even when your change had nothing to do with the failure.

I hit this on snok: the node had a stale IPv6 default gateway left over from an ISP prefix change. Every NNCP failed verification on the dead IPv6 gateway and was rolled back. One policy even ended up permanently stuck in Pending / MaxUnavailableLimitReached. If oc get nnce shows a policy cycling or stuck, check the nmstate handler logs for probe errors before blaming your YAML:

oc -n openshift-nmstate logs ds/nmstate-handler | grep -i probe

Fix the broken gateway first, then apply the bridge.

5. Don’t put the mirror port on OVN’s bridges

It’s tempting to add the NIC to an OVS bridge and map it with an OVN localnet network, since that’s how secondary VLAN networks usually work in OpenShift Virtualization. For mirrored traffic it’s the wrong tool: OVN’s logical switch forwards by destination MAC and won’t hand foreign unicast frames to your VM ports. Keep the mirror on a plain Linux bridge, separate from br-ex and any OVN-mapped bridges.

6. Live migration

The VMs can only run on nodes that have the mirror cable plugged in. On a single-node cluster that’s automatic. On multi-node clusters, the resourceName annotation keeps the scheduler honest, but live migration to a node without the mirror feed won’t work. Set evictionStrategy: None (or plan for the sensors to stop during node maintenance).

Verifying end to end

1. On the node, check that mirrored traffic is arriving:

   tcpdump -i enp65s0f0 -nn -c 10
   

2. Check that both taps are on the bridge:

   bridge link show master br-mirror
   

3. Inside each guest, run tcpdump -i eth1 -nn and make sure you see traffic between other hosts. That’s the proof the hub is flooding and not just passing broadcasts.
4. Generate a known flow from a mirrored host (for example curl a known site) and confirm both sensors see it.

If the guests only see broadcasts, ARP and multicast but no unicast between other hosts, the bridge is still learning MACs. Check ageing_time on br-mirror and make sure it’s 0.

Summary

To share one switch mirror port with several OpenShift Virtualization VMs:

1. Put the mirror NIC in a Linux bridge with mac-ageing-time: 0 and multicast snooping off, so the bridge acts as a hub. Turn off GRO/LRO on the NIC.
2. Expose the bridge with a bridge CNI NetworkAttachmentDefinition that has no IPAM.
3. Give each VM a second bridge interface on that network and set it to promiscuous mode in the guest.
4. Remember that a SPAN destination is usually receive-only. Use a separate normal port if your VMs need to send.

It takes three short YAML files, and it avoids the destination-MAC filtering that defeats OVN, SR-IOV and macvtap for this use case.

The backstory: my home lab ran HAProxy bolted onto pfSense for years, routing traffic across a dozen-plus internal apps and a few OpenShift clusters. I wanted it out of the firewall and into its own purpose-built, disposable container: non-root by design, with automatic Let’s Encrypt certs via DNS-01 (Porkbun), and a live REST API for config changes instead of editing files by hand.

A few things I’m happy with:

• 🔒 Runs as a non-root user, no --cap-add needed for privileged ports. A Linux file capability handles that.
• 📦 Trimmed the image from 459MB down to 244MB by ripping out a redundant build stage.
• 🔁 Certs issue and renew independently per domain (or grouped as SANs when a service needs it), so one DNS hiccup doesn’t take down the whole fleet.

It’s live on GitHub if you want to see how it’s built: https://github.com/nr3v0/magic-proxy-ubi

… Not what this post is about, follow the Red Hat and Microsoft documentation and then consult with your certified Red Hat Solutions Architect or Consultant.

Now that I have a windows node

• <WARNING> Administering a windows node is highly discouraged and will probably result in an unsupported configuration. In order to install nerdctl, log into the necessary Windows Node and switch to PowerShell.

ssh win2022

Microsoft Windows [Version 10.0.20348.1070]
(c) Microsoft Corporation. All rights reserved.

administrator@WIN2022 C:\Users\Administrator> powershell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\Users\Administrator>

Append to System PATH - Permanently

Borrowing commands from a friendly blogger we need to append a new entry to the system PATH. Permanently Modify System Path Run these:

rem Show Current Path
PS C:\Users\Administrator> $ENV:PATH
rem Save Current Path to a variable
PS C:\Users\Administrator> $oldpath = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path

rem Add a new folder for our binary
PS C:\Users\Administrator> mkdir C:\bin
rem Change directory to the new folder
PS C:\Users\Administrator> cd C:\bin
PS C:\bin> $newpath = "$oldpath;C:\bin"

rem Update System Path
PS C:\bin> Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $newPath

rem Verify the Path
PS C:\bin> (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).Path

rem Reboot node - or close out of your shells and relogin
Restart-Computer -Force

Download and install nerdctl.exe

With all that yak shaving out of the way we can get to it. Find the latest windows release (nerdctl-x.y.z-windows-amd64.tar.gz) nerdctl-1.0.0-windows-amd64.tar.gz nerdctl releases

rem start Powershell and change to the bin folder
rem Download the release file.
PS C:\bin> curl https://github.com/containerd/nerdctl/releases/download/v1.0.0/nerdctl-1.0.0-windows-amd64.tar.gz -o nerdctl-1.0.0-windows-amd64.tar.gz

rem Extract nerdctl.exe from the compressed file
PS C:\bin> tar -xzvf nerdctl-1.0.0-windows-amd64.tar.gz
x nerdctl.exe

rem Delete the compressed file
PS C:\bin> rm nerdctl-1.0.0-windows-amd64.tar.gz

rem Test a nerdctl.exe command
PS C:\bin> nerdctl.exe ps
CONTAINER ID    IMAGE    COMMAND    CREATED    STATUS    PORTS    NAMES

Celebrate

You can now interact with your containers on the windows node.

The Windows Server 2022 Core VM we were working with had a http proxy set but the OpenShift cluster was in an environment that did not need a proxy. Using the top google hits we were unable to clear the “advproxy” settings.

C:\> netsh winhttp show proxy
Current WinHTTP proxy settings:

    Direct access (no proxy server).

C:\> netsh winhttp show advproxy
{
        "Proxy":        "localhost:8080",
        "ProxyBypass":  "*"
}

Using netsh winhttp reset proxy only cleared the proxy settings. It did NOT clear the advproxy settings.

The solution

Create a JSON file with the settings required for netsh winhttp set advproxy command.

PS C:\Users\Administrator> New-Item proxy.json
    Directory: C:\Users\Administrator

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        11/14/2022   4:05 PM              0 proxy.json

PS C:\Users\Administrator> Set-Content .\proxy.json '{"Proxy": "", "ProxyBypass": "", "AutoconfigUrl": "", "AutoDetect": false}'
PS C:\Users\Administrator> Get-Content proxy.json
{"Proxy": ""}

Then execute the netsh command below:

netsh winhttp set advproxy setting-scope=machine settings-file=proxy.json

Current WinHTTP advanced proxy settings:

{
}

Thank You

I do not know the difference between the two settings, just know they both influence the system.

This feels like a match made in heaven for organizations that are slowly consuming their monoliths or who have decided on Microsoft technologies for their datacenter. Either way, Windows Node support has come a long way in the last 2 years.

I must preface this discussion with the assumption that “everyone knows that Linux and WIndows are different”. Not in the way “milk is different from water” but more along the lines of steak is different from broccoli. Sure, there are many ways the interaction feels similar but would probably wouldn’t steam a nice steak or smother broccoli in steak sauce(not unreasonable). Unfortunately, Kubernetes exposes all the ways the two are different. Some of this is probably to the history of Kubernetes being developed on Linux, using Linux Containers, with the strong drive of kernel developers to maintain a stable ABI. With WIndows 11 and Windows Server 2022 Microsoft has finally been able to address the ABI/Kernel version issues it suffered from and make the experience closer to Linux. But there are so many other differences which I won’t address.

Next, many organizations are adopting Nutanix in their datacenters and wish to run WIndows Containers on OpenShift on Nutanix. OpenShift 4.11+ on Nutanx is a wonderful combination and is fully supported. Windows Container support in Kubernetes is coming along nicely. Windows Server 2022 makes windows containers usable outside of AKS or Docker Desktop. And the Red Hat Windows Machine Config Operator(WMCO) automates the process of adding a Windows VM to a Kubernetes cluster with the OpenShift Machine api.

The WMCO is written to deal with differences between Linux hosts and Windows hosts and is written and maintained by Red Hat and the community. OpenSSH needs to be installed and enabled on the Windows host as this is the medium which files are transferred, services are created and the host gets configured. As an operator, WMCO has its own lifecycle and support policy. As of this writing it only supports platform types of (none, aws and vsphere).

At this point in time the only way to get a Windows Node in a cluster installed on Nutanix is with BYOH and some unsupported steps. Again THIS METHOD IS UNSUPPORTED and support is on its way. Here is a rough outline too add a host assuming the SDN is OVNKubernetes and Hybrid Networking was configured at cluster install.

Steps

1. Install cluster with networking.type: OVNKuberenetes and HybridNetworking
2. Run prerequisites
   • install wmco namespace
   • install wmco operator
   • create cloud-private-key secret
     • https://docs.openshift.com/container-platform/4.11/windows_containers/enabling-windows-container-workloads.html
3. Create a VM and install Windows 2022 (core or ui)
4. Make modifications to match the environment.
5. Install OpenSSH
6. Create SSH Key for the Administrator user and add to administrator_authorized_user
   • $powershell.exe get-content $env:ProgramData\ssh\administrators_authorized_keys
7. Verify you can log in from your bastion host
8. OpenPort 10250
   • powershell.exe New-NetFirewallRule -DisplayName "ContainerLogsPort" -LocalPort 10250 -Enabled True -Direction Inbound -Protocol TCP -Action Allow -EdgeTraversalPolicy Allow
9. Run Windows SysPrep similar to the vsphere steps 8-9
10. Shutdown VM and Create a Template or Disk Image
11. Clone VM
12. Start Cloned VM
13. Verify IP is unique and verify reverse DNS (PTR) is correct
14. Change hostname of Cloned VM
    • powershell.exe Rename-Computer -NewName <valid-name> -Restart
    • reboot
15. As documented in the BYOH instructions Create windows-instances ConfigMap
16. Initially you may see the node come into the cluster correctly. However, once the kubelet starts it selects the “hybrid interface” IP and tries to join the cluster with this incorrect IP.
17. Once you see the node added, Ready and Unscheduleable
    • ssh to windows host
    • Display the binpath of the kubelet service
      • sc.exe qc kubelet
    • Copy binpath from above command and add --node-ip=<correct_ip> to the end
      • sc.exe config binPath= "c:\k\kubelet.exe --config=c:\k\kubelet.conf --bootstrap-kubeconfig=c:\k\bootstrap-kubeconfig --kubeconfig=c:\k\kubeconfig --cert-dir=c:\var\lib\kubelet\pki\ --windows-service --logtostderr=false --log-file=C:\var\log\kubelet\kubelet.log --register-with-taints=os=Windows:NoSchedule --node-labels=node.openshift.io/os_id=Windows --container-runtime=remote --container-runtime-endpoint=npipe://./pipe/containerd-containerd --resolv-conf= --cloud-provider= --v=3 --node-ip=192.168.20.104"
    • Restart kubelet
      • powershell restart-service kubelet
18. Now you should see 3+ Approved CSR’s and your node should become Ready and Scheduleable

Troubleshooting:

Pending CSR’s?

Indicates the reverse DNS record is missing or incorrect

Node stuck UnScheduleable?

Node IP isn’t in the same subnet as the linux nodes. Or Firewall is blocking something

Appendix - Known issues with Windows Containers on Nutanix

Kubelet chooses the wrong IP to add to the cluster

The kubelet code is failing to lookup the correct IP from the nutanix machine api implementation and falls into the “grab a random IP”, which obviously always grabs the wrong one. This is what step #16 works around. Providing an argument to the kubelet for the node-ip, bypassing auto-descovery.

Nutanix Machine Provider API hardcodes “Legacy BIOS” as the boot mode

Customers wishing to boot their kubernetes VM’s with UEFI will need to wait for the resolution of Issue 29

Nutanix Machine Provider API only accepts “Image”

Do not confuse this with a “Template”. An Image in Nutanix defines just the disk volume (think VMDK or qcow2) and does not contain any VM definitions. There is not an issue written to address this.

But I want to try a MachineSet

You can try. Clone your existing machine set and make the correct modification. However, the kubelet still chooses the incorrect IP and changes to the service get overwritten by WMCO and the machine-api

Where can I follow the work

RFE-3354 machine-api-provider-nutanix issue 29

Working with Windows Containers has provided me tremendous education. Learning PowerShell has shown me how far the windows realm has come in the form of CLI management. It still seems to be lacking in a few areas, and I really don’t like the extremely verbose syntax nor the single dash arguments. But I digress. This post is about solving a single problem. The one problem with endless obtuse workarounds.

Problem

Editing text files…from a windows CLI (cmd.exe, PowerShell, etc).

Solution

Find the Microsoft equivalent of Vim. Is that Notepad.exe? Wordpad.exe? Get-Contents? Fail, Fail, Mostly-Fail

My Solution

Install vim-console; an extremely powerful text editor that has proven to be the staple of all *nix users around the world for decades.

Note: There are many articles similar to How to Edit Files with a Real PowerShell Text Editor that show how to use Chocolaty to install Vim, Emacs or Nano. However, most of my use cases don’t allow Chocolaty in their datacenters. So, we have to install vim the hard way.

Steps

1. SSH to your server.

   Don’t know how to setup ssh on Windows? I will point you at their documentation: Getting Started OpenSSH for Windows

2. Verify you are in PowerShell.

   Type the command powershell or pwsh and hit enter.

   

   Look for the PS in front of your command prompt.

   

3. Download vim-console artifacts from NLUUG - dunno where I found this

   wget https://ftp.nluug.nl/pub/vim/pc/vim90rt.zip -o vim90rt.zip
   wget https://ftp.nluug.nl/pub/vim/pc/vim90w32.zip -o vim90w32.zip
   

4. Extract archives

   expand-archive .\vim90w32.zip -DestinationPath .
   expand-archive  -Force .\vim90rt.zip -DestinationPath .
   

5. Run installer

   cd vim\vim90
   .\install.exe
   

   Change settings as needed and say d for do it.

   

6. Run and Profit

   • Simply type vim and hit enter
   • To exit, the key sequence Esc-:-q-<enter> will exit.

Conclusion

I didn’t go into a lot of detail but hopefully this helps with the steps for using VIM.

-Nate

I find myself constantly using multiple terminals on the same system. The default bash history is a good start, but not enough for my needs. After endless searching, testing, ignoring, deleting, and procrastinating I finally found a solution that works for me.

1. Session Isolation
   • I don’t need new history items available across sessions
   • Typically the work I am preforming is isolated but similar enough that having commands updated between sessions is both obnoxious and unhelpful.
2. Custom Timestamp
   • I like seeing a particular timestamp format in my history output (helps me remember what I was doing at that time)
3. Dedupe
   • Like I mentioned above, I don’t need commands shared across terminal windows
   • Also, I don’t like duplicate uses of command in my history
   • Sure I may lose the first, or prior, occurrence(s) of a command. But I only require the last use.
4. Write on session exit
   • I don’t want to lose my history to a kill -i <shell_pid>
   • I also don’t want to lose my history if I close the wrong session tab
   • dedupe only needs to happen at this point
5. Use only linux bash commands
   • I don’t want to download some fancy utility.
   • use the fewest number of commands
6. Unlimited history
   • computers are big enough and fast enough to handle this
   • new versions of bash allow an unlimited history

~/.bashrc

HISTCONTROL=ignorespace:ignoredups:erasedups
shopt -s histappend
HISTSIZE=-1
HISTFILESIZE=-1
HISTTIMEFORMAT="[%b %d,%Y %T] "
PROMPT_COMMAND="history -a;${PROMPT_COMMAND}"

/etc/profile.d/00-dedup-bash-history-on-exit.sh

function deduphistory {
  echo "deduphistory"
  history -w ~/.bash_history.dedup_backup
  history -n; history -a
  tac ~/.bash_history | awk '!x[$0]++' | tac > ~/.bash_history_new_dedup
  history -cr ~/.bash_history_new_dedup
  history -w
}
# cleanup bash history on exit
trap deduphistory EXIT

Profit

Watch your history get appended and deduped on exit. No more lost history (mostly) And be happy

Files available as gists

https://gist.github.com/therevoman/a7532af575479f0a2b5edfbb0bef8212

Originally posted on GitHub: https://github.com/therevoman/nate-revo-blog/issues/2

My computer adventure started in the early (19)80’s with the TI-99 and various Tandy Computers. I was the family “Technical Advisor” even though I really only knew just a little more than they did. Fascinated by what they could offer I was exposed to the Nintendo, ColecoVision, Adam, Amiga 500, Packard Bell, and a Cray Supercomputer. Being just a youngling I was not allowed to play with the beta edition of Windows NT running on it, but it inspired me to keep my mind open and always be learning.

In the early ’90s I entered the BBS scene and opened a BBS on my parents home computer. Being the encouraging parents they are, and after seeing the first phone bill, they purchased a 2nd line on which I could run so their “ears would be preserved”.

In the mid 90’s I was introduced to this novel Operating System named Linux and downloaded the first release on 14 1.44mb floppy diskettes. Having only 10 disks available I downloaded the first 10 disks during a University class. That night I began the installation, leaving the computer on and praying there would be no power outage I overwrote the first few diskettes to get the remaining installation images. Success! I was presented with a shell prompt of some sort and my lack of knowledge kept me from doing anything more than startx and xeyes.

At University I was exposed to Caldera Linux, which I will still hail as the “easiest and most user-friendly” installation ever available. Too bad their business ethics did what they did (enter groklaw).

With only one “Java” programming class under my belt I began my career as a web-developer with the goal of using Java on the AS/400 (iseries (is it now series-i?!)) and learning how to install WebSphere v1 and v3 on the platform’s IFS.

The next 18 years I moved around, working in various government and financial institutions. This gave me exposure to many different organizational structures and to this day helps me relate to the struggles many IT staff experience. And yes, I believe I can understand the frustration of infrastructure administrators AND developers.

In 2017 I discovered containers through docker and OpenShift v3. I had no idea how big of an impact this would have on my career. In fact, I was so impressed by the technology I jokingly sent in a resume in the hopes I could catch the train. 3 weeks later I arrived at Red Hat Tower in Raleigh, NC for my new hire orientation. A few weeks after that I was sent to 2 weeks of OpenShift Bootcamp and was released into the wild as a Kubernetes (aka OpenShift) admin. The journey from Java Developer to OpenShift infrastructure Administrator was enormous, one I am glad I made, and almost overbearing at times. Thanks to the many great people at Red Hat and my great family I was able to made the transition.

Now, I spent my free time playing with my son, building many things and dabbling in all sorts of technology. This blog is dedicated to my Technology adventures and I hope it helps someone else.

-Nate

HTTP error (404 - Not Found): Unit with ID “da2cba1f-a856-4a7e-8df0-b6f4101587af” could not be found.

I’m not exactly sure, but it seems the bios/efi information is the same across all the boxes. To get arround the error with subscription-manager I logged directly into https://access.redhat.com/management/systems and added the system manually. Once I had a UUID I ran subscription-manager again to register the system.

subscription-manager register --consumerid=<new-uuid>

Happy cheap labbing.

At least I could get RHV installed on my main Lab box (I’ll enumerate my lab later). It’s big enough and fast enough that I can run RHV and put my control plane components on it (Ceph MON’s, OpenStack Director, etc) when I get the time. So I started a RHV 4.3 installation.

It started pretty harmless. Follow the pre-reqs, make sure you hardware is right, and set the repos. Link: https://access.redhat.com/products/red-hat-virtualization/#getstarted

Because of a DNS quirk I introduced last year when wiring NetworkManager, libvirt and dnsmasq together, I had to restart the install more than a dozen times. One involved having multiple IP’s returned for the hypervisor host, which was fixed in my local DNS provider (unbound). I swapped back and forth between the cockpit and command line installs but it kept failing with an ssh error. After a lot of googling I landed on the host_key_checking=false ansible setting, and after some lucky guesswork I determined it belonged in the file /usr/share/ovirt-hosted-engine-setup/ansible/ansible.cfg. And while I was in there I added a few performance tweaks, which left it looking like this:

## Modify Ansible to work properly for a lab?
/usr/share/ovirt-hosted-engine-setup/ansible/ansible.cfg
[defaults]
inventory = hooks/inventory
host_key_checking = false
forks = 20

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey
pipelining = true

This got me past the jankiness above.

The next error was a very strange parse error dumped by the vdsm-tool for the libvirt module. The error was super misleading (complicated by my idiocy), so I traced into the code and found a parsing bug in my /etc/libvirt/qemu.conf file. A long while back I had added nvram settings so I could boot UEFI vm’s. The solution was to move the closing bracket off its own line.

Error Message:

# vdsm-tool configure --module libvirt
Checking configuration status...
libvirt is not configured for vdsm yet
Traceback (most recent call last):
File "/usr/bin/vdsm-tool", line 220, in main
return tool_command[cmd]["command"](*args)
File "/usr/lib/python2.7/site-packages/vdsm/tool/__init__.py", line 40, in wrapper
func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/tool/configurator.py", line 124, in configure
if _should_configure(c, pargs.force)]
File "/usr/lib/python2.7/site-packages/vdsm/tool/configurator.py", line 318, in _should_configure
if not _validate(c) and not configure_allowed:
File "/usr/lib/python2.7/site-packages/vdsm/tool/configurator.py", line 85, in _validate
return getattr(module, 'validate', lambda: True)()
File "/usr/lib/python2.7/site-packages/vdsm/tool/configurators/libvirt.py", line 72, in validate
return _isSslConflict()
File "/usr/lib/python2.7/site-packages/vdsm/tool/configurators/libvirt.py", line 115, in _isSslConflict
qconf_p.read(confutils.get_file_path('QCONF', FILES))
File "/usr/lib/python2.7/site-packages/vdsm/tool/configfile.py", line 266, in read
io.StringIO(u'[root]\n' + f.read())
File "/usr/lib64/python2.7/ConfigParser.py", line 324, in readfp
self._read(fp, filename)
File "/usr/lib64/python2.7/ConfigParser.py", line 546, in _read
raise e
ParsingError: File contains parsing errors:
[line 760]: u']\n'

These are the offending lines in /etc/libvirt/qemu.conf:

nvram = [
"/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd"
]

Here is the fix:

nvram = [ "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd" ]

The rest of the install proceeded without incident! WooHoo!!! I have RHV!

Ugh, what’s that Gluster thing?