2020-03-07

ubuntuでstrongswanしてlogを設定するとapparmorに殴られた

study network

ubuntuの鎧が硬かったのでメモっておく．charonはデフォでsyslogにlogを吐いてくるのでややウザく，/var/log/charon/logにログ出力しようとしたのが事の発端．
例えばこんな風にして軽率に/var/log配下にlogを書こうとする

jp7fkf@lab1:~$ cat /etc/strongswan.d/charon-logging.conf
charon {
    filelog {
        default = 1
        /var/log/charon.log {
            path = /var/log/charon.log
            time_format = %b %e %T
            ike_name = yes
            append = no
            default = 1
            flush_line = yes
            time_add_ms = yes
            net = 2
            tls = 2
            knl = 2
        }
    }
}

しかしlogは/var/log/charon.logには出てこない． syslogを覗いてみるとこんなmessageが．

Feb 27 22:17:48 lab1 kernel: [ 8212.229038] audit: type=1400 audit(1582820000.905:38): apparmor="DENIED" operation="mknod" profile="/usr/lib/ipsec/charon" name="/var/log/charon.log" pid=16202 comm="charon" requested_mask="c" denied_mask="c" fsuid=0 ouid=0

apparmor="DENIED"
と言われているのでapparmorで/var/log/charon.logへの書き込みが弾かれてるっぽい．

適切に権限をあげればいい気がする．
apparmorに/var/log/charon.log*への書き込み権限を加える．
/etc/apparmor.d/配下でcharonのapparmor settingを見ていじる．

jp7fkf@lab1:~$ sudo cat /etc/apparmor.d/usr.lib.ipsec.charon
# ------------------------------------------------------------------
#
#   Copyright (C) 2016 Canonical Ltd.
#
#   This program is free software; you can redistribute it and/or
#   modify it under the terms of version 2 of the GNU General Public
#   License published by the Free Software Foundation.
#
#   Author: Jonathan Davies <jonathan.davies@canonical.com>
#           Ryan Harper <ryan.harper@canonical.com>
#
# ------------------------------------------------------------------

#include <tunables/global>

/usr/lib/ipsec/charon flags=(attach_disconnected) {
  #include <abstractions/base>
  #include <abstractions/nameservice>
  #include <abstractions/authentication>
  #include <abstractions/openssl>
  #include <abstractions/p11-kit>

  capability ipc_lock,
  capability net_admin,
  capability net_raw,

  # allow priv dropping (LP: #1333655)
  capability chown,
  capability setgid,
  capability setuid,

  # libcharon-extra-plugins: xauth-pam
  capability audit_write,

  # libstrongswan-standard-plugins: agent
  capability dac_override,

  capability net_admin,
  capability net_raw,

  network,
  network raw,

  /bin/dash                 rmPUx,

  # libchron-extra-plugins: kernel-libipsec
  /dev/net/tun              rw,

  /etc/ipsec.conf           r,
  /etc/ipsec.secrets        r,
  /etc/ipsec.*.secrets      r,
  /etc/ipsec.d/             r,
  /etc/ipsec.d/**           r,
  /etc/ipsec.d/crls/*       rw,
  /etc/opensc/opensc.conf   r,
  /etc/strongswan.conf      r,
  /etc/strongswan.d/        r,
  /etc/strongswan.d/**      r,
  /etc/tnc_config           r,

  /proc/sys/net/core/xfrm_acq_expires   w,

  /run/charon.*             rw,
  /run/pcscd/pcscd.comm     rw,

  /usr/lib/ipsec/charon     rmix,
  /usr/lib/ipsec/imcvs/     r,
  /usr/lib/ipsec/imcvs/**   rm,

  /usr/lib/*/opensc-pkcs11.so rm,

  /var/lib/strongswan/*     r,

  /var/log/charon.log*       rw, # ここに/var/log/charon.log*のrw権限をつける．

  # for using the ha plugin (LP: #1773956)
  @{PROC}/@{pid}/net/ipt_CLUSTERIP/ r,
  @{PROC}/@{pid}/net/ipt_CLUSTERIP/* rw,

  # Site-specific additions and overrides. See local/README for details.
  #include <local/usr.lib.ipsec.charon>
}

あとはapparmorをreloadさせる．
sudo systemctl reload apparmor.service

するとlogがちゃんと出てくる．

jp7fkf@lab1:~$ cat /var/log/charon.log
Feb 27 22:22:44.144 00[DMN] Starting IKE charon daemon (strongSwan 5.6.2, Linux 5.0.0-1031-gcp, x86_64)
Feb 27 22:22:44.454 00[KNL] known interfaces and IP addresses:
Feb 27 22:22:44.454 00[KNL]   lo
Feb 27 22:22:44.454 00[KNL]     127.0.0.1
Feb 27 22:22:44.454 00[KNL]     ::1
(omit...)

めでたし．

2020-03-07

systemdにpython scriptをservice登録してdaemon化する

study software

もう十分internetに知見が転がっていると思うが，pythonをdarmon化したくなったので自分のためにもメモっておく．適当にservice fileを書いてやればいいだけ．もはやpythonだからとか関係ない．ただshebangは書いておかないとダメかも．permission的にはもちろん実行権限をつけておく．

ex:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

(omit...)

service fileを書く

% cat /etc/systemd/system/testscript_py.service
[Unit]
Description = testscript_py

[Service]
ExecStart = /home/jp7fkf/testscript_py/run.py
Restart = always
Type = simple

[Install]
WantedBy = multi-user.target

daemon-reloadでservice fileを読み込む

% sudo systemctl daemon-reload

読まれた．適宜 sudo systemctl enable testscript_py やらをすればいい

[jp7fkf@lab1 18:11:52] ~
% systemctl status testscript_py
● testscript_py.service - testscript_py
   Loaded: loaded (/etc/systemd/system/testscript_py.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2020-03-04 00:47:02 JST; 17h ago
 Main PID: 1857 (python3)
    Tasks: 12 (limit: 4434)
   CGroup: /system.slice/testscript_py.service
           └─1857 /home/jp7fkf/.pyenv/versions/3.8.1/bin/python3 /home/jp7fkf/testscript_py/run.py

2020-01-25

OVAをqcow2にconvertする

study

OVAイメージをqcow2にconvertして利用したくなったので調べてみた． VMを構成するためのファイルというのは，いくつかの種類がある．よく聞くのがova, ovf, qcow2などではないだろうか．今回はこれらのファイルについての意味合いの理解を深め，仮想イメージを変換することを試してみる．

OVAファイルとは何者か

OVAとは"Open Virtual Appliance"のアクロニムである．これはOVF，仮想ディスクイメージ，その他付随するファイルをtarでarchiveしたものだと思えばよいようだ．

ではOVFとは何か

OVFとは"Open Virtualization Format"のアクロニム．

Open Virtualization Format（OVFとOVA）を参照すると，OVFパッケージに内包されるファイルには下記のような種類があるようだ．

記述子: 記述子ファイルにより，その仮想マシンの仮想ハードウェアが定義される．このファイルには仮想ディスク，サービス，およびゲストオペレーティングシステムに関する記述や，ライセンス契約書（EULA），アプライアンス内の仮想マシンの起動および停止手順，サービスのインストール手順などの情報が含まれる．記述子ファイルの拡張子は，.ovf である．
マニフェスト: パッケージに含まれる各ファイルのSHA-1ダイジェスト値で，パッケージの破損を検出する．マニフェストファイルの拡張子は，.mf である．
署名: パッケージに含まれるX.509証明書の公開キーで署名されたマニフェストのダイジェスト値で，パッケージの所有者の検証に使用される．署名ファイルの拡張子は，.cert である．
仮想ディスク: OVFは，ディスクイメージの形式についての仕様ではない．OVFパッケージには仮想ディスクを構成するファイルを含むが，その形式は仮想ディスクをエクスポートした仮想化製品により異なる．XenServerで作成するOVFパッケージでは，Dynamic VHD形式のディスクイメージが使用される．VMware製品やVirtual BoxのOVFパッケージでは，ストリーム最適化のVMDK形式が使用される．

このうち，記述子と呼ばれるファイルがOVFファイルを指す．このファイルがどういう役割を持っているかというと，仮想マシンのconfiguration(CPU, Memory, NIC, GuestOSの種類, etc...)，ストレージ，ネットワーク構成，その他メタデータ等がxmlで記述されている． OVFファイルがあれば，あるVM(群)の構成を構築することが可能ということである．ただしこのOVFにはディスクイメージは含まないため，その他の手段でディスクイメージを提供する必要がある．具体的にはvmdk等が挙げられる．

OVFパッケージにはこの記述子(ovf)とイメージ(vmdk, etc...)のほか，マニフェストファイル(.mf)と署名ファイル(.cert)が含まれることが多いようである．マニフェストファイルはovaに含まれる各ファイルのsha-1ダイジェストが格納されており，ファイルの破損を検知する目的で利用されるようだ．また，署名ファイルにはマニフェストファイルをX.509証明書の公開キーで署名したダイジェスト値が記載されており，所有者(distributor)の検証が行えるようになっているようだ．

OVFファイルとOVFパッケージは似て非なるもので，OVFパッケージにはOVFファイルが常に含まれるが，場合によっては上記のマニフェスト，署名，仮想ディスクが含まれ，これをtarでarchiveしたものがOVAと呼ばれるファイルとなる．なので，tar -xf hoge.ova すると上記の4つのファイル群が得られることになる．

vmdkをqcow2に変換する

ovaがtarアーカイブであることがわかったので，tar -xf <.ova> で展開すると，仮想ディスクイメージが得られるはずである．仮想ディスクイメージの種類にはいくつか存在するが，メジャーなものはqemu-img コマンドを用いて変換することができるようだ．

qemu-imgコマンドはqemu-utilsに内包されているようなので，installされていない場合は sudo apt install qemu-utils 等で入手できそうである．

qemu-imgを用いた変換の容量は下記の通りである．

qemu-img convert [-f format] [-O output_format] <input_image_file> <output_image_file>

formatに利用できる引数はそれぞれ下記の通り
- QCOW2 (KVM, Xen): qcow2
- QED (KVM): qed
- raw: raw
- VDI (VirtualBox): vdi
- VHD (Hyper-V): vpc
- VMDK (VMware): vmdk

これで相互にイメージが変換できるはずだ．

実際にやってみる

vyosのOVAイメージ vyos-1.1.7-amd64-signed.ova を例に上記の変換までを実施してみる．

[jp7fkf@lab1]$ ls
vyos-1.1.7-amd64-signed.ova
[jp7fkf@lab1]$
[jp7fkf@lab1]$ tar -xvf vyos-1.1.7-amd64-signed.ova
VyOS-1.1.7-signed.ovf
VyOS-1.1.7-signed.mf
VyOS-1.1.7-signed.cert
VyOS-1.1.7-signed-disk1.vmdk
[jp7fkf@lab1]$ ls
vyos-1.1.7-amd64-signed.ova  VyOS-1.1.7-signed-disk1.vmdk  VyOS-1.1.7-signed.ovf
VyOS-1.1.7-signed.cert       VyOS-1.1.7-signed.mf

tar archiveを展開するとovf, vmdk, mf, certの4つのファイルが出てくる．

マニフェストファイルを見てみよう

[jp7fkf@lab1]$ cat VyOS-1.1.7-signed.mf
SHA1(VyOS-1.1.7-signed.ovf)= f075a5ea7058d16734c82fbe0097abebacc769ce
SHA1(VyOS-1.1.7-signed-disk1.vmdk)= a9e83ab5c700a920d657d6d1b52f8dda3e160efc

想定通り，OVFとvmdkについてのsha-1 digestが記載されている．certはmfから生成されるので，ここには載っていない．

certを見てみると，

[jp7fkf@lab1]$ cat VyOS-1.1.7-signed.cert
SHA1(VyOS-1.1.7-signed.mf)= c76c44df048078b1528a7e0174ab98d2add165c4c91c8d25b58c4561c1368b5c2ccf523e1f614b40dde75a99323ccce3c3a836dae7eabbb5b22964d517c34d2eefbabd70878437aee9694113ca55469e50625b3c7c3efeee93b53ffb036f2ca8aaa8f0d7653cdcb0b38a7063a38769f5ab1c132bedf96b10b2cb11670c5a5b5ad01097f8317fa0a6e9b8882eea36b87bfa3c1efa4adaf7866fba72a03a4651d060f079a4163965a89accb507a671fb228c70e9fb6d1d4e1d8b1a9087a4d80ca73b9b34687624847d6dfd609a917932d088c5f3fbbea9ab49f0b2499fb0f3ea428cc3500513f67159acb03e1326bbcb2b678ee296f0a9ba0a5f9365e198aadb54da0c28678610c2a11ec6126f01cb1985bd44d90293be99bb37c55e41f1483c68c6f2e84604e9e3637af6cb963007616adf9960515e4ef79becbac152c153cc587cb6bcdd2ef580561504d2f8fc7122f5c68072ea8dcd4cba434db4cb53cbbc01130edea3c9a73e114c7ff95709f4a5087e6b026bac3c0ce6b0ebae28c7a15d9767ed3b24b702c1586a70c32bf5eb62aa3ec9c18bea2e43e32b36b023bf592166f10f8079d155eac41010bca46b954c9c9741c934af3fcb4dda687e53fa813d271bcac3ad86117b3238ed512d729edb2fe1ec224bc0a71b872b5ed7d485efb95d374057daddfdb8365593b7cf182e698c88656900dd67e3c14a73dd225786b277
-----BEGIN CERTIFICATE-----
MIIGADCCBOigAwIBAgIQIgpv5xjEXl10TvXTz+syBTANBgkqhkiG9w0BAQsFADB1
MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjEpMCcGA1UECxMg
U3RhcnRDb20gQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkxIzAhBgNVBAMTGlN0YXJ0
Q29tIENsYXNzIDMgT2JqZWN0IENBMB4XDTE2MDMwODA1MTIzNVoXDTE5MDMwODA1
MTIzNVowYTELMAkGA1UEBhMCRVMxDzANBgNVBAgMBk1hZHJpZDERMA8GA1UEBwwI
QWxjb3Jjb24xFjAUBgNVBAoMDVNlbnRyaXVtIFMuTC4xFjAUBgNVBAMMDVNlbnRy
aXVtIFMuTC4wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDfWjvOnc3m
FWnFdKphQxS1IicZVN0YOoWrDCKbl89hLQD5w76JjWfmLm9yceIjgKIhHvmbHNWH
FpXhTpzLw12YvfrUzjhFTePNDrdHPLbXSaV5F2xkArPLrASFxOVwXJ3kHJPjfs1U
NvG6FEcwfUWC+t2Fzsdto9nI5U44qtU7d9C60Mprx3qnpej4mzgQsc61jCPatARN
cjA/xxC51zxlB4mX+599SZHewX8HyyZtXYPCcq47LzeQqqhqub0LUFKpe47sjzv/
6qzTpz62O7OsKRp1zecCHA0pwvtuFWydQMksvTqSb6l/vXZ1Yf45bvuEpJjePS1N
aMXGa5cpvzSY67YKS3XnaNmnCY2d3T3d0X624CZ+PB7LcSGbcaOH9pMglsASCEBy
Hd0RuTcjWUQaxU5zqcl3H+HmP92sMjum5g6irX47oTi3CvJnK7TdFahNJdyepejo
cVfaVGXf2BnHD6veupjMGBCpecYAvRDBnqrctYBKwNt39+PdA9Cnkn4n9Vx+GFX2
xdP6Wnw+ot9m2JbBeBr4A+BXqb4MXd9eCM2mRBduNliAdN0QrLtJLVPL/1NPUd5D
3XgezSwArkHZduxg2eqdaZd1l0Tn2dNyXOxl7wasSjyENBJR9ovlhDas0eA8Ll5V
nsQc0mV6s+QCzwCoW3lQc8G9hN27sLc9gwIDAQABo4IBnjCCAZowDgYDVR0PAQH/
BAQDAgeAMB8GA1UdJQQYMBYGCCsGAQUFBwMDBgorBgEEAYI3PQEBMAkGA1UdEwQC
MAAwHQYDVR0OBBYEFOfLPzVM5pWho/73a/g9s8ZaPvksMB8GA1UdIwQYMBaAFGZ6
ns2cc4ZqaaCu+oy7GI8I7NUEMG0GCCsGAQUFBwEBBGEwXzAkBggrBgEFBQcwAYYY
aHR0cDovL29jc3Auc3RhcnRzc2wuY29tMDcGCCsGAQUFBzAChitodHRwOi8vYWlh
LnN0YXJ0c3NsLmNvbS9jZXJ0cy9zY2EuY29kZTMuY3J0MDYGA1UdHwQvMC0wK6Ap
oCeGJWh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL3NjYS1jb2RlMy5jcmwwIwYDVR0S
BBwwGoYYaHR0cDovL3d3dy5zdGFydHNzbC5jb20vMFAGA1UdIARJMEcwCAYGZ4EM
AQQBMDsGCysGAQQBgbU3AQIEMCwwKgYIKwYBBQUHAgEWHmh0dHA6Ly93d3cuc3Rh
cnRzc2wuY29tL3BvbGljeTANBgkqhkiG9w0BAQsFAAOCAQEAuyMGjwa/FgoZzEgL
5w78V5y9oUDrEN1rRgglcMc80Tvcv5Nv3JrS1LoUw8GXVMHkcxl+g86atWUNBXuD
5HZf9xRLBknOw5L6szOm0Wdptdcc42Iu27tUCYY2Lf8e1Qo6JzUXP/Z2q+vZEqDh
l/moBXxIss26tUsU4JzZe9mpgdZt3lxeD/NT13rWhcDVvuRVlUtysT4LBGm0j+5q
K8sMkYSDwoL8lj8t4gnhq9C3Sz6lf3kETJpNfZIBvZuwqcWtGrK3gFqojTljBfzu
pK+M/HqsgMkFVWOaY/x/SrcWPRX7b/hZkbcpXBkkTDwDegpwtlWrL8NuM9OfFoMn
/5oB2w==
-----END CERTIFICATE-----

mfのsha-1 digestと署名が記載があることがわかる．

続いてOVFをはじめの20行についてのみ見てみると，

[jp7fkf@lab1]$
[jp7fkf@lab1]$ cat VyOS-1.1.7-signed.ovf  | head -n 20
<?xml version="1.0" encoding="UTF-8"?>
<!--Generated by VMware VirtualCenter Server, User: FLEXIMOVIL.LOCAL\syncer, UTC time: 2016-04-10T19:20:43.849526Z-->
<Envelope vmw:buildId="build-3339084" xmlns="http://schemas.dmtf.org/ovf/envelope/1" xmlns:cim="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <References>
    <File ovf:href="VyOS-1.1.7-signed-disk1.vmdk" ovf:id="file1" ovf:size="245504000"/>
  </References>
  <DiskSection>
    <Info>Virtual disk information</Info>
    <Disk ovf:capacity="4" ovf:capacityAllocationUnits="byte * 2^30" ovf:diskId="vmdisk1" ovf:fileRef="file1" ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized" ovf:populatedSize="254869504"/>
  </DiskSection>
  <NetworkSection>
    <Info>The list of logical networks</Info>
    <Network ovf:name="management.gs.es">
      <Description>The management.gs.es network</Description>
    </Network>
  </NetworkSection>
  <vmw:IpAssignmentSection ovf:required="false" vmw:protocols="IPv4" vmw:schemes="dhcp">
    <Info>Supported IP assignment schemes</Info>
  </vmw:IpAssignmentSection>
  <VirtualSystem ovf:id="VyOS-1.1.7">
[jp7fkf@lab1]$

xml形式であることがわかる．細かく見ていくと，Disk SectionやNetwork Sectionなど，VMの構成に関する情報が記載されていることが見えてくる．

ではvmdkをqcow2に変換してみよう

[jp7fkf@lab1]$ qemu-img convert -f vmdk -O qcow2 VyOS-1.1.7-signed-disk1.vmdk VyOS-1.1.7-signed-disk1.qcow2
[jp7fkf@lab1]$ ls
vyos-1.1.7-amd64-signed.ova  VyOS-1.1.7-signed-disk1.qcow2  VyOS-1.1.7-signed.mf
VyOS-1.1.7-signed.cert       VyOS-1.1.7-signed-disk1.vmdk   VyOS-1.1.7-signed.ovf
[jp7fkf@lab1]$ file vyos-1.1.7-amd64-signed.ova
vyos-1.1.7-amd64-signed.ova: POSIX tar archive
[jp7fkf@lab1]$ file VyOS-1.1.7-signed-disk1.qcow2
VyOS-1.1.7-signed-disk1.qcow2: QEMU QCOW Image (v3), 4294967296 bytes
[jp7fkf@lab1]$

想定通り，qcow2に変換できていそうだということがわかる．

まとめ

OVAファイルが何者か，また内包されるOVF, 仮想イメージ，マニフェスト，署名についての意味合い，実態についてまとめた．
仮想ディスクイメージファイルの相互の変換をqemu-imgコマンドを利用して行うことができることを述べた．
実際にvyosのOVAファイルを用いてコマンドを実行し，変換が行われていることを確認した．

References

2020-01-07

RFC 3074を読んでみた話

network study

RFC 3074 - DHC Load Balancing Algorithm

RFC3074はDHCPのロードバランシングアルゴリズムについての手法が書かれたものである． keaのHA構成について調べてるところ，RFCを読んでみたくなり，和訳してみたので載せてみます．特に意味はありませんが，何か参考になればと思います．

概要

この文章ではロードバランシングのアルゴリズムを提案します．これは，はじめの設定以外の情報交換を必要とせずに，複数のサーバのうちのどれがクライアントに対して応答するかを決定するものです．

このサーバ選択アルゴリズムは，複数のDHCPサーバがクライアントにサービス提供を行う際，サーバで計算するクライアントのMACアドレスのハッシュをもとにします．この提案手法は，複数のDHCPサーバがネットワーク上で動作する時，既存のDHCPクライアントになんの変更の必要もなしに，効率的なサーバ選択を提供します．同様の手法はBOOTP relayのような転送エージェントを利用する場合に，ターゲットサーバを選択する場合にも提案されています．

1. 序論

このプロトコルはもともと，DHCP Failoverプロトコルのうち，特にロードバランシング最適化のために生まれました．著者らはのちに，冗長化されたDHCPサーバと，それらにパケットを転送するBOOTP relay agentの動作を最適化するために利用できることに気がつきました．この提案では，それぞれのサーバでクライアントの負荷割合をあらかじめ設定することができます．これは，決定論的ハッシュアルゴリズムを用いて行われ，似た特性を持つ他のプロトコルに容易に適用できます．

2. 用語

このセクションでは，多くのIETF プロトコル使用で共通な一般的な必要用語と，この文章で導入される用語の両方について議論します．

2.1 必要用語

"MUST", "MUST NOT", "REQUIERD", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", "OPTIONAL"のキーワードについては，RFC 2119に記載されている通りに解釈します．

2.2 ロードバランシング用語

このドキュメントでは下記の用語について紹介します．

Service Delay, SD
- ロードバランシングのパラメータの一つで，クライアントを無視する代わりにロードバランシングに参加しているサーバによる遅延サービスを許可する遅延時間．
Hask Bucket Assignments, HBA
- ロードバランシングスキームに参加するサーバに一連のHack Backet値を割り当てる構成ディレクティブ．
Server ID, SID
- 参加しているサーバの一つを指定するために利用されるID．DHCPのコンテキストでは，このSIDはIPアドレスか，サーバのDNS nameである．
Service Transaction, ST
- サーバがクライアントにサービス提供もしくは拒否をする一連の情報交換．例: DHCPサーバとクライアントで交換される，DISCOVER/OFFER/REQUEST/ACK メッセージはService Transactionである．
Service Transaction ID, STID
- ロードバランシングに利用される個別のクライアントリクエストの属性．

3. バックグラウンドと外部要件

DHCPクライアントはDHCPサーバにコンタクトするためにUDP broadcastを利用するため，クライアントのDHCPDISCOVERメッセージは1台以上のサーバで受信される可能性があります．ブロードキャストを受け取った全てのサーバはクライアントに応答し，クライアントにどのサーバを利用するかを選ばせるでしょう．

BOOTP relay agentを利用する場合，クライアントのブロードキャストは一般にすべての設定されたサーバに転送，もしくは再ブロードキャストされるという同様の非効率性が存在します．

後述の最適化により，それぞれのトランアクションに対して,"サービスする"/"サービスしない"振る舞いをすることでサーバを選択することができます．転送エージェントは転送先を決定するために同様の計算で動作できます．

いずれのケースでも，どちらが応答するかネゴシエーションすることなくサーバの選択がおこなわれます．

どのクライアントが次にリクエストをするかを予測することはほぼ不可能であるため，このアプローチは本質的には確率的なものです．短時間のうちにおいては，実際のところ特定のサーバによってサービスされるクライアントの割合は指定した割合から逸脱する可能性は十分あります．リクエストの数が上昇すると，それぞれのサーバで処理される実際の負荷の割合は設定された割合に近づくでしょう．

4. 概要

DHCPサーバはClient Identifierが存在する場合はそれを必ずSTIDとして利用します．もしClient Identifierオプションがない場合，DHCPパケットのhlenフィールドはハッシュ化されたデータの長さのために利用し，chaddrはハッシュ化されたデータでなければなりません．Client Identifierもしくはchaadrの最も始めの16bytesが利用されます．

この手案はこのSTIDをsection 6に示す関数を用いてハッシュ化した値に対応させます．そして，結果のハッシュ値を使って，どのサーバがリクエストに応答するべきか，もしくは転送するべきターゲットを決定する必要があります．

このハッシュ関数は0-255のハッシュ値を生成し，ランダムなSTIDや，パターン性のあるSTIDシーケンスにかなり均等なHack Backet分布を生成します．リソースアロケーションはそれぞれのサーバに特定のハッシュ値セットを割り当てることで実現します．

リクエストのSTIDハッシュが自身に割り当てられたハッシュ値セットのうちにマッチした場合にのみ，サーバはリクエストに応答します．

どのサーバに割り当てされていないHack Backetは一部のクライアントSTは完全に無視されることになります(いくつかの状況では，これは好ましい結果になるかもしれません)．STIDはユニークである必要はありませんが，負荷を各サーバに分散するために十分ばらけているべきです．

HBAは他のプロトコルでカプセル化されて送信されるかもしれません．例: e-mail, DHCP Failover protocol option.

DHCPサーバの実装では，ロードバランシングは行われているがレスポンスが不可能であるもしくは十分なアドレスがないサーバを扱うための設定をオプションで設定することができます．

もし，client requestのsecsフィールドの値が0でなければ，この機能を提供するDHCPサーバはこの値を利用するべきです．いくつかのクライアントはこのsecsフィールドの実装が正しくないため，DHCPサーバは，通常は応答しないクライアントトランザクションのはじめのインスタンスを見続ける場合があります．もしサーバが過去に記録されたリクエストと同様のトランザクションIDをもったリクエストを受信した場合，もしくは2番目のpacketのsecsフィールドが0の場合，DHCPサーバは，最初とその次のクライアントリクエストまでの経過時間をsecsフィールドの代わりに利用することができます．

5. 動作

5.1 設定

この設定のステップは利用可能なサーバにハッシュ値を割り当てることから成ります．これは一つ以上のHBAが提供されていることで実現します．それらは，設定ファイルや，Windows NTレジストリ，EEPROMなどから構成されます．あるいは，HBAは決められたアルゴリズムを利用して割り当てることもできます．例: 奇数はserverA, 偶数はserverBが応答するなど．

5.2 サーバ向けのHBA

あるサーバを設定する時，32オクテットのシンプルなビットマップ形式を利用するべきです． HBAビットマップはじめのオクテットは0-7のHBA値で表現され，次は8-15，のように表現され，第32オクテットで248-255の値を表現します．それぞれのオクテットで，LSBはそのオクテットの最小のHBA値を表現します．

それぞれのHBAのビットは一つの取りうるハッシュ値にひもづきます．もしあるビットがビットマップにセットされていたら，それはSTIDのハッシュ値に対応して生成されたそれぞれのクライアントリクエストに応答しなければならないことを意味します．

たとえば，もしサーバには下記のような32オクテットのHBAが設定されていた場合，

            FF FF FF FF FF FF 00 00 ( 0   - 63 )
            FF FF FF FF FF FF FF FF ( 64 - 127 )
            00 00 00 00 00 00 00 00 (128 - 191 )
            00 00 00 00 00 00 00 00 (192 - 255 )

そのサーバはSTIDのハッシュがHack Backetの0-47,と64-127の値のであるあらゆるクライアントリクエストに対して応答しなければなりません．

5.3 遅延サービスパラメータ

遅延サービスパラメータはオプションです．

もしパラメータが設定されていない場合，HBAは厳格に"サービスする"/"サービスしない"というポリシを構成します．

もしパラメータが設定されていた場合，HBAやSTID hashに基づく特定のリクエストに対して応答することになっていないサーバが，クライアントのはじめの試行から, S 秒間経過したのちに応答することを許可します．

サーバはBOOTPヘッダのsecsフィールドをサービスを利用しようとした時からの経過時間として利用しても構いませんし，他の手法で繰り返しのリクエストを追跡しても構いません．

5.4 フォワーダ向けのHBA

BOOTP relayといった転送エージェントを設定する際，Server-IDとHack Backet値のペアからなるHBAが利用できます．

ここで，Server ID(SID)は特定のHack Backetを担うサーバを指定します．その転送エージェントはSTIDが特定のハッシュ値を生成した各クライアントリクエストを，SIDで指定されたサーバに転送します．このサーバIDは何らかのユニークなサーバ属性(例: IP address, DNS nameなど)が利用でき，リレーエージェントの動作上，意味があるものです．

フォワーダは1つ以上のサーバにパケットを転送するように設定されているかもしれません．たとえば，BOOTP relayは互いでDHCP Failover Protocolが動作している2つのprimary-backupサーバペアの間で負荷を分割するように設定さえうります．

このありがちな転送エージェント(例: BOOTP realy)の設定ファイルとして，下記のようなものがあります

192.33.43.11 192.33.43.12: 0..24;
192.33.43.13:  25..55;
192.33.43.15:  56..128;
192.33.43.16: 129 130 131 200..202;

上の設定は4つのHBAからなっています．はじめのHBAの例は次のように読み取れます: "STIDのハッシュ値が0-24となるすべてのクライアントリクエストは192.133.43.11と192.33.43.12の両方のサーバに転送されます"

6. ロードバランシング用ハッシュ関数

下記のハッシュ関数はピアソンハッシュとして知られるアルゴリズムのC言語の実装です．このピアソンハッシュアルゴリズムはもともと[PEARSON: The Communications of the ACM Vol.33, No. 6 (June 1990), pp. 677-680.] で公開されました．

このハッシュ関数はそれぞれのキーバイトに配列検索とXOR演算を必要とし，計算機的に軽量なものです．この提案を使うためには，相互運用性のあるすべての実装はこのハッシュ関数を使用する必要があります．ミキシングテーブルの値のセットを下記に示します:

/* A "mixing table" of 256 distinct values, in pseudo-random order. */

unsigned char  loadb_mx_tbl[256] ={
251, 175, 119, 215, 81, 14, 79, 191, 103, 49, 181, 143, 186, 157,  0,
232, 31, 32, 55, 60, 152, 58, 17, 237, 174, 70, 160, 144, 220, 90, 57,
223, 59,  3, 18, 140, 111, 166, 203, 196, 134, 243, 124, 95, 222, 179,
197, 65, 180, 48, 36, 15, 107, 46, 233, 130, 165, 30, 123, 161, 209, 23,
97, 16, 40, 91, 219, 61, 100, 10, 210, 109, 250, 127, 22, 138, 29, 108,
244, 67, 207,  9, 178, 204, 74, 98, 126, 249, 167, 116, 34, 77, 193,
200, 121,  5, 20, 113, 71, 35, 128, 13, 182, 94, 25, 226, 227, 199, 75,
27, 41, 245, 230, 224, 43, 225, 177, 26, 155, 150, 212, 142, 218, 115,
241, 73, 88, 105, 39, 114, 62, 255, 192, 201, 145, 214, 168, 158, 221,
148, 154, 122, 12, 84, 82, 163, 44, 139, 228, 236, 205, 242, 217, 11,
187, 146, 159, 64, 86, 239, 195, 42, 106, 198, 118, 112, 184, 172, 87,
2, 173, 117, 176, 229, 247, 253, 137, 185, 99, 164, 102, 147, 45, 66,
231, 52, 141, 211, 194, 206, 246, 238, 56, 110, 78, 248, 63, 240, 189,
93, 92, 51, 53, 183, 19, 171, 72, 50, 33, 104, 101, 69, 8, 252, 83, 120,
76, 135, 85, 54, 202, 125, 188, 213, 96, 235, 136, 208, 162, 129, 190,
132, 156, 38, 47, 1, 7, 254, 24, 4, 216, 131, 89, 21, 28, 133, 37, 153,
149, 80, 170, 68, 6, 169, 234, 151
};

unsigned char loadb_p_hash(
        const unsigned char *key,       /* The key to be hashed */
        const int len )                 /* Key length in bytes  */
{
unsigned char hash  = len;
int i;

        for (i=len ; i > 0 ;  )
            hash = loadb_mx_tbl  [ hash ^ key[ --i ] ];

        return( hash );
}

int accept_service_request(
        const unsigned char HBA[32],    /* The hash bucket bitmap */
        const unsigned char *key,       /* The service transaction id*/
        const int len  )                /* length of the above */
{
unsigned char hash = loadb_p_hash(key,len);
int index          = (hash >> 3) & 31;
int bitmask        = 1 << (hash & 7);

        /* return 1 if we should service this transaction */
        return((HBA[index] & bitmask) != 0);
}

7. セキュリティの考察

この提案の内容とそれ自身は，セキュリティを提供せず，既存のセキュリティに影響を与えることもありません．もし，HBAの内容がいずれかのサーバを構成するプロセスの一部としてネットワークに送信されることがある場合，このアルゴリズムを利用するサーバは，そのメッセージの改ざんの脅威から守る責任があります．HBAの改ざんによって，いくつかもしくはすべてのクライアントへのサービスの提供不可を招くためです．

8. 参考

[FAILOVR] Kinnear, K,, Droms, R., Rabil, G., Dooley, M., Kapur, A., Gonczi, S. and B. Volz, "DHCP Failover Protocol", Work in Progress.

[PEARSON] The Communications of the ACM Vol.33, No. 6 (June 1990), pp. 677-680.

[RFC2131] Droms, R., "Dynamic Host Configuration Protocol", RFC 2131, March 1997.

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels," BCP 14, RFC 2119, March 1997.

9. 謝辞

ピアソンハッシュの著者であるPeter K. Pearsonに感謝します．彼は何の制約なしに彼のアルゴリズムを利用することを快く許可してくれました．

この提案は1999年2月，CISCO Systemsで開かれたFailover Protocolの会議中にTed LemonがMAC Addressを1ビットにハッシュ化する独特のアイディアが肝となっています．Rob StevensはFailover Protocolの目的を超えて，このアルゴリズムの有用性を示唆しました．

議論中にコメントをいただいた，Ralph Droms，Kim Kinnear，Mark Stapp，Glenn Waters，Greg Rabil，Jack Wongらに多大なる感謝をいたします．

まとめ

KeaのHAやISC DHCPのFailoverではRFC 3074 - DHC Load Balancing Algorithmが利用されているようなのでとりあえず和訳してみた．
STIDからピアソンハッシュアルゴリズムに基づいて0-255のハッシュ値をとり，そのハッシュ値とHBA bitmapを照らし合わせ，その値とANDを取った時trueである場合，自らが応答するという仕組みになっている．
このHBAは設定ファイルやその他の方法で，冗長化するサーバ同士で適切に構成する必要があるが，その手法はこのRFCには含まない．

References

RFC 3074 - DHC Load Balancing Algorithm

2020-01-05

Kea DHCPのHigh-Availabilityを検証する - (2)Act-ActのLoad-Balancing構成を組む

network software

ゴール

Kea DHCPを用いてAct-ActのLoad-Balancing High-Availability構成を組み，動作させることができること．

Kea DHCPでLoad-BalancibngモードのHA構成を組む

Introduction

前回，Kea DHCPでhot-standby構成のHA構成を組む方法について紹介をした．今回はhot-standbyではないもう一つの方法であるload-balancingモードでのHA構成を組んでみる．

検証条件は前回と同様(dockerfile, IPアドレスや本記事で紹介しないconfigなどすべて)であるので，kea自体の紹介や，環境構築等については前回の記事を参照してほしい．本記事ではhot-standbyとload-balancingモードの違いについて特に焦点を当てて紹介する．

configuration

早速configを行なっていく． load-balancingモードの利用に伴い，modeや各peer config内のroleが異なってくる． load-balancingモードでkea_1をprimary, kea_2をsecondaryとする場合，configは下記のようになる．

this-server-name: kea_1 //(2台目は"kea_2")
mode: load-balancing
heartbeat-delay: 10000
max-response-delay: 60000
max-ack-delay: 10000
max-unacked-clients: 0
"peers": [
  {
    "name": "kea_1",
    "url": "http://172.18.0.3:8080/",
    "role": "primary",
    "auto-failover": true
  },
  {
    "name": "kea_2",
    "url": "http://172.18.0.2:8080/",
    "role": "secondary",
    "auto-failover": true
  }
]

上記を踏まえてconfigを作成し，各サーバに適用して動作を確認してみよう．
たとえば私の検証では下記のようなconfigをkea_1に投入した．kea_2もほぼ同様であるが，"this-server-name" 等を適切に変更する．

root@085fd121c5c6:~# cat /usr/local/etc/kea/kea-dhcp4.conf
{
"Dhcp4": {
    "interfaces-config": {
        "interfaces": ["eth0"],
        "dhcp-socket-type": "raw",
        "outbound-interface": "use-routing"
    },
    "control-socket": {
        "socket-type": "unix",
        "socket-name": "/tmp/kea-dhcp4-ctrl.sock"
    },
    "lease-database": {
        "type": "memfile",
        "persist": true,
        "name": "/usr/local/etc/kea/kea-leases4.csv",
        "lfc-interval": 1800
    },
    "expired-leases-processing": {
        "reclaim-timer-wait-time": 10,
        "flush-reclaimed-timer-wait-time": 25,
        "hold-reclaimed-time": 3600,
        "max-reclaim-leases": 100,
        "max-reclaim-time": 250,
        "unwarned-reclaim-cycles": 5
    },

    "renew-timer": 1200,
    "rebind-timer": 2400,
    "valid-lifetime": 3600,

    "option-data": [
        {
            "name": "domain-name-servers",
            "data": "8.8.8.8, 8.8.4.4"
        },
        {
            "name": "default-ip-ttl",
            "data": "0xf0"
        }
    ],

    "hooks-libraries": [
        {
            "library": "/usr/local/lib/hooks/libdhcp_lease_cmds.so",
            "parameters": { }
        },
        {
            "library": "/usr/local/lib/hooks/libdhcp_ha.so",
            "parameters": {
                "high-availability": [ {
                    "this-server-name": "kea_1",
                    "mode": "load-balancing",
                    "heartbeat-delay": 10000,
                    "max-response-delay": 10000,
                    "max-ack-delay": 5000,
                    "max-unacked-clients": 5,
                    "peers": [
                      {
                        "name": "kea_1",
                        "url": "http://172.18.0.3:8080/",
                        "role": "primary",
                        "auto-failover": true
                      },
                      {
                        "name": "kea_2",
                        "url": "http://172.18.0.2:8080/",
                        "role": "secondary",
                        "auto-failover": true
                      }
                    ]
                } ]
            }
        }
    ],

    "reservation-mode": "disabled",
    "host-reservation-identifiers": [ "hw-address" ],

    "subnet4": [
          {
                "id": 1,
                "subnet": "172.18.0.0/16",
                "pools": [
                    { "pool": "172.18.0.100 - 172.18.0.250" }
                ],
                "option-data": [
                    {
                        "name": "routers",
                        "data": "172.18.0.1"
                    }
                ]
          }
    ]
},

"Logging":
{
  "loggers": [
    {
        "name": "kea-dhcp4",
        "output_options": [
            {
                "output": "/usr/local/var/log/kea-dhcp4.log",
                "maxver": 8,
                "maxsize": 204800,
                "flush": true
            }
        ],
        "severity": "DEBUG",
        "debuglevel": 99
    }
  ]
}
}

まずはkea_1でkeaを起動する．今回，起動にはkeactrlコマンドを用いた．起動後にRESTAPIでha-heartbeatを送信していることと，それぞれのサーバのステータスを見てみよう．

# kea_1
root@085fd121c5c6:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.3:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 172.18.0.3 port 8080: Connection refused
root@085fd121c5c6:~# keactrl start
INFO/keactrl: Starting /usr/local/sbin/kea-dhcp4 -c /usr/local/etc/kea/kea-dhcp4.conf
INFO/keactrl: Starting /usr/local/sbin/kea-ctrl-agent -c /usr/local/etc/kea/kea-ctrl-agent.conf
root@085fd121c5c6:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.3:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   191  100   138  100    53  34500  13250 --:--:-- --:--:-- --:--:-- 63666
[
  {
    "arguments": {
      "date-time": "Wed, 20 Nov 2019 06:56:46 GMT",
      "state": "waiting"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@085fd121c5c6:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.3:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   196  100   143  100    53  11916   4416 --:--:-- --:--:-- --:--:-- 17818
[
  {
    "arguments": {
      "date-time": "Wed, 20 Nov 2019 06:56:50 GMT",
      "state": "partner-down"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]

起動後，kea_1はstateが waitingから，partner-downにシフトしていることが見受けられる． kea_1を起動し，kea_2を起動していない場合，kea_1はこの状態(partner-down)に収束する．

ここでkea_2を起動する．

# kea_2
root@05e1a4a4476e:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.2:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 172.18.0.2 port 8080: Connection refused
root@05e1a4a4476e:~# keactrl start
INFO/keactrl: Starting /usr/local/sbin/kea-dhcp4 -c /usr/local/etc/kea/kea-dhcp4.conf
INFO/keactrl: Starting /usr/local/sbin/kea-ctrl-agent -c /usr/local/etc/kea/kea-ctrl-agent.conf
root@05e1a4a4476e:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.2:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--   100   191  100   138  100    53  46000  17666 --:--:-- --:--:-- --:--:-- 95500
[
  {
    "arguments": {
      "date-time": "Wed, 20 Nov 2019 06:57:12 GMT",
      "state": "waiting"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@05e1a4a4476e:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.2:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--   100   191  100   138  100    53  34500  13250 --:--:-- --:--:-- --:--:-- 63666
[
  {
    "arguments": {
      "date-time": "Wed, 20 Nov 2019 06:57:20 GMT",
      "state": "waiting"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@05e1a4a4476e:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.2:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--   100   189  100   136  100    53  12363   4818 --:--:-- --:--:-- --:--:-- 21000
[
  {
    "arguments": {
      "date-time": "Wed, 20 Nov 2019 06:57:32 GMT",
      "state": "ready"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@05e1a4a4476e:~# curl -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http://172.18.0.2:8080/ | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--   100   198  100   145  100    53   7250   2650 --:--:-- --:--:-- --:--:--  9900
[
  {
    "arguments": {
      "date-time": "Wed, 20 Nov 2019 06:57:36 GMT",
      "state": "load-balancing"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]

ログを見てみる．

# kea_1
root@085fd121c5c6:/# grep INFO /usr/local/var/log/kea-dhcp4.log | grep ha-hooks
2019-11-20 06:56:39.814 INFO  [kea-dhcp4.ha-hooks/387] HA_CONFIGURATION_SUCCESSFUL HA hook library has been successfully configured
2019-11-20 06:56:39.814 INFO  [kea-dhcp4.ha-hooks/387] HA_INIT_OK loading High Availability hooks library successful
2019-11-20 06:56:39.884 INFO  [kea-dhcp4.ha-hooks/387] HA_LOCAL_DHCP_DISABLE local DHCP service is disabled while the kea_1 is in the WAITING state
2019-11-20 06:56:39.884 INFO  [kea-dhcp4.ha-hooks/387] HA_SERVICE_STARTED started high availability service in load-balancing mode as primary server
2019-11-20 06:56:50.323 INFO  [kea-dhcp4.ha-hooks/387] HA_STATE_TRANSITION server transitions from WAITING to PARTNER-DOWN state, partner state is UNDEFINED
2019-11-20 06:56:50.323 INFO  [kea-dhcp4.ha-hooks/387] HA_LEASE_UPDATES_DISABLED lease updates will not be sent to the partner while in PARTNER-DOWN state
2019-11-20 06:56:50.323 INFO  [kea-dhcp4.ha-hooks/387] HA_LOCAL_DHCP_ENABLE local DHCP service is enabled while the kea_1 is in the PARTNER-DOWN state
2019-11-20 06:57:33.825 INFO  [kea-dhcp4.ha-hooks/387] HA_STATE_TRANSITION server transitions from PARTNER-DOWN to LOAD-BALANCING state, partner state is READY
2019-11-20 06:57:33.825 INFO  [kea-dhcp4.ha-hooks/387] HA_LEASE_UPDATES_ENABLED lease updates will be sent to the partner while in LOAD-BALANCING state
root@085fd121c5c6:/#

# kea_2
root@05e1a4a4476e:/# grep INFO /usr/local/var/log/kea-dhcp4.log | grep ha-hooks
2019-11-20 06:57:11.496 INFO  [kea-dhcp4.ha-hooks/512] HA_CONFIGURATION_SUCCESSFUL HA hook library has been successfully configured
2019-11-20 06:57:11.496 INFO  [kea-dhcp4.ha-hooks/512] HA_INIT_OK loading High Availability hooks library successful
2019-11-20 06:57:11.528 INFO  [kea-dhcp4.ha-hooks/512] HA_LOCAL_DHCP_DISABLE local DHCP service is disabled while the kea_2 is in the WAITING state
2019-11-20 06:57:11.528 INFO  [kea-dhcp4.ha-hooks/512] HA_SERVICE_STARTED started high availability service in load-balancing mode as secondary server
2019-11-20 06:57:23.295 INFO  [kea-dhcp4.ha-hooks/512] HA_STATE_TRANSITION server transitions from WAITING to SYNCING state, partner state is PARTNER-DOWN
2019-11-20 06:57:23.295 INFO  [kea-dhcp4.ha-hooks/512] HA_LEASE_UPDATES_DISABLED lease updates will not be sent to the partner while in SYNCING state
2019-11-20 06:57:23.295 INFO  [kea-dhcp4.ha-hooks/512] HA_SYNC_START starting lease database synchronization with kea_1
2019-11-20 06:57:23.306 INFO  [kea-dhcp4.ha-hooks/512] HA_LEASES_SYNC_LEASE_PAGE_RECEIVED received 0 leases from kea_1
2019-11-20 06:57:23.308 INFO  [kea-dhcp4.ha-hooks/512] HA_SYNC_SUCCESSFUL lease database synchronization with kea_1 completed successfully in 12.135 ms
2019-11-20 06:57:23.308 INFO  [kea-dhcp4.ha-hooks/512] HA_STATE_TRANSITION server transitions from SYNCING to READY state, partner state is PARTNER-DOWN
2019-11-20 06:57:23.308 INFO  [kea-dhcp4.ha-hooks/512] HA_LEASE_UPDATES_DISABLED lease updates will not be sent to the partner while in READY state
2019-11-20 06:57:34.860 INFO  [kea-dhcp4.ha-hooks/512] HA_STATE_TRANSITION server transitions from READY to LOAD-BALANCING state, partner state is LOAD-BALANCING
2019-11-20 06:57:34.860 INFO  [kea-dhcp4.ha-hooks/512] HA_LEASE_UPDATES_ENABLED lease updates will be sent to the partner while in LOAD-BALANCING state
2019-11-20 06:57:34.860 INFO  [kea-dhcp4.ha-hooks/512] HA_LOCAL_DHCP_ENABLE local DHCP service is enabled while the kea_2 is in the LOAD-BALANCING state
root@05e1a4a4476e:/#

これらをみてみると，起動直後，これらのHAは下記のような順序で状態遷移していることがわかる．

2019-11-20 06:56:39.884:: kea_1: waiting     , kea_2: UNDEFINED
2019-11-20 06:56:50.323:: kea_1: partner-down, kea_2: UNDEFINED
2019-11-20 06:57:11.528:: kea_1: partner-down, kea_2: waiting
2019-11-20 06:57:23.295:: kea_1: partner-down, kea_2: syncing
2019-11-20 06:57:23.308:: kea_1: partner-down, kea_2: ready
2019-11-20 06:57:33.825:: kea_1: load-balancing,  kea_2: ready
2019-11-20 06:57:34.860:: kea_1: load-balancing , kea_2: load-balancing

このそれぞれのstateの意味はkeaのdocumentを参照していただくのがbestだと思うが，前回の記事でも簡単に説明しているので参考になるかもしれない．

load-balancingモードでのDHCPリース動作

HA構成を組むことができていることが確認できたので，この状態でDHCPのアドレスリースを行い，HA構成に受けるアドレスリース時の振る舞いを確認していこうと思う．

DHCPリースを模擬するために，前回同様，dhtestを利用した．

load-balancingモードでHA構成が組まれたサーバ群が接続されているNW上に存在するdhtest用VMで，下記の通りコマンドを実行する．

./dhtest -i <interface> -m <mac_addr>

mac_addrはdhtestホストから送出するDHCPパケットのchaddrフィールドに入るMACアドレスと思えばよい．これを数回繰り返し実施した時の振る舞いを見てみる． dhtestホストでは下記のようにアドレスリースが模擬できているようである．

[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:33
DHCP discover sent   - Client MAC : 00:00:00:11:22:33
DHCP offer received  - Offered IP : 172.18.0.100
DHCP request sent  - Client MAC : 00:00:00:11:22:33
DHCP ack received  - Acquired IP: 172.18.0.100
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:34
DHCP discover sent   - Client MAC : 00:00:00:11:22:34
DHCP offer received  - Offered IP : 172.18.0.101
DHCP request sent  - Client MAC : 00:00:00:11:22:34
ADHCP ack received  - Acquired IP: 172.18.0.101
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:35
DHCP discover sent   - Client MAC : 00:00:00:11:22:35
DHCP offer received  - Offered IP : 172.18.0.102
DHCP request sent  - Client MAC : 00:00:00:11:22:35
DHCP ack received  - Acquired IP: 172.18.0.102
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:36
DHCP discover sent   - Client MAC : 00:00:00:11:22:36
DHCP offer received  - Offered IP : 172.18.0.103
DHCP request sent  - Client MAC : 00:00:00:11:22:36
DHCP ack received  - Acquired IP: 172.18.0.103
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:37
DHCP discover sent   - Client MAC : 00:00:00:11:22:37
DHCP offer received  - Offered IP : 172.18.0.104
DHCP request sent  - Client MAC : 00:00:00:11:22:37
DHCP ack received  - Acquired IP: 172.18.0.104
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:38
DHCP discover sent   - Client MAC : 00:00:00:11:22:38
DHCP offer received  - Offered IP : 172.18.0.105
DHCP request sent  - Client MAC : 00:00:00:11:22:38
DHCP ack received  - Acquired IP: 172.18.0.105
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:39
DHCP discover sent   - Client MAC : 00:00:00:11:22:39
DHCP offer received  - Offered IP : 172.18.0.106
DHCP request sent  - Client MAC : 00:00:00:11:22:39
DHCP ack received  - Acquired IP: 172.18.0.106
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:40
DHCP discover sent   - Client MAC : 00:00:00:11:22:40
DHCP offer received  - Offered IP : 172.18.0.107
DHCP request sent  - Client MAC : 00:00:00:11:22:40
DHCP ack received  - Acquired IP: 172.18.0.107
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:41
DHCP discover sent   - Client MAC : 00:00:00:11:22:41
DHCP offer received  - Offered IP : 172.18.0.108
DHCP request sent  - Client MAC : 00:00:00:11:22:41
DHCP ack received  - Acquired IP: 172.18.0.108
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:42
DHCP discover sent   - Client MAC : 00:00:00:11:22:42
DHCP offer received  - Offered IP : 172.18.0.109
DHCP request sent  - Client MAC : 00:00:00:11:22:42
DHCP ack received  - Acquired IP: 172.18.0.109
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:43
DHCP discover sent   - Client MAC : 00:00:00:11:22:43
DHCP offer received  - Offered IP : 172.18.0.110
DHCP request sent  - Client MAC : 00:00:00:11:22:43
DHCP ack received  - Acquired IP: 172.18.0.110
[root@a50202e240fb dhtest-master]#

この時，kea_1 とkea_2 ホストでパケットキャプチャを行うと．

## kea_1
root@085fd121c5c6:/# tcpdump -i eth0 port 67 or port 68 -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^[[O06:59:13.512449 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:33 (oui Ethernet), length 251, xid 0x2960582c, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:33 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:13.552499 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:33 (oui Ethernet), length 263, xid 0x2960582c, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:33 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.100
      Server-ID Option 54, length 4: kea_2.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:18.869778 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:34 (oui Ethernet), length 251, xid 0x31c40ac1, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:34 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:18.871641 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    085fd121c5c6.bootps > 172.18.0.101.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x31c40ac1, Flags [none]
    Your-IP 172.18.0.101
    Client-Ethernet-Address 00:00:00:11:22:34 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: Offer
      Server-ID Option 54, length 4: 085fd121c5c6
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:18.872159 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:34 (oui Ethernet), length 263, xid 0x31c40ac1, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:34 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.101
      Server-ID Option 54, length 4: 085fd121c5c6
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:19.878008 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    085fd121c5c6.bootps > 172.18.0.101.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x31c40ac1, Flags [none]
    Your-IP 172.18.0.101
    Client-Ethernet-Address 00:00:00:11:22:34 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: ACK
      Server-ID Option 54, length 4: 085fd121c5c6
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:22.559720 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:35 (oui Ethernet), length 251, xid 0x4c120d1c, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:35 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:22.563249 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:35 (oui Ethernet), length 263, xid 0x4c120d1c, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:35 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.102
      Server-ID Option 54, length 4: kea_2.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:26.252357 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:36 (oui Ethernet), length 251, xid 0x671e7d0f, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:36 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:26.265846 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:36 (oui Ethernet), length 263, xid 0x671e7d0f, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:36 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.103
      Server-ID Option 54, length 4: kea_2.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:38.855959 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:40 (oui Ethernet), length 251, xid 0x16e71cb, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:40 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:38.857611 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    085fd121c5c6.bootps > 172.18.0.107.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x16e71cb, Flags [none]
    Your-IP 172.18.0.107
    Client-Ethernet-Address 00:00:00:11:22:40 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: Offer
      Server-ID Option 54, length 4: 085fd121c5c6
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:38.857778 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:40 (oui Ethernet), length 263, xid 0x16e71cb, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:40 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.107
      Server-ID Option 54, length 4: 085fd121c5c6
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:39.859876 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    085fd121c5c6.bootps > 172.18.0.107.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x16e71cb, Flags [none]
    Your-IP 172.18.0.107
    Client-Ethernet-Address 00:00:00:11:22:40 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: ACK
      Server-ID Option 54, length 4: 085fd121c5c6
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:41.995310 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:41 (oui Ethernet), length 251, xid 0x44f61b6e, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:41 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:41.996738 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    085fd121c5c6.bootps > 172.18.0.108.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x44f61b6e, Flags [none]
    Your-IP 172.18.0.108
    Client-Ethernet-Address 00:00:00:11:22:41 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: Offer
      Server-ID Option 54, length 4: 085fd121c5c6
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:46.075757 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:42 (oui Ethernet), length 251, xid 0x4da855db, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:42 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:46.078611 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:42 (oui Ethernet), length 263, xid 0x4da855db, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:42 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.109
      Server-ID Option 54, length 4: kea_2.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:49.416302 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:43 (oui Ethernet), length 251, xid 0x39e5fa04, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:43 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:49.418794 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:43 (oui Ethernet), length 263, xid 0x39e5fa04, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:43 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.110
      Server-ID Option 54, length 4: kea_2.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
^C
20 packets captured
32 packets received by filter
12 packets dropped by kernel
root@085fd121c5c6:/#

## kea_2
root@05e1a4a4476e:/# tcpdump -i eth0 port 67 or port 68 -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:59:13.512513 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:33 (oui Ethernet), length 251, xid 0x2960582c, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:33 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:13.551115 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    05e1a4a4476e.bootps > 172.18.0.100.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x2960582c, Flags [none]
    Your-IP 172.18.0.100
    Client-Ethernet-Address 00:00:00:11:22:33 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: Offer
      Server-ID Option 54, length 4: 05e1a4a4476e
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:13.552540 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:33 (oui Ethernet), length 263, xid 0x2960582c, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:33 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.100
      Server-ID Option 54, length 4: 05e1a4a4476e
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:14.559982 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    05e1a4a4476e.bootps > 172.18.0.100.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x2960582c, Flags [none]
    Your-IP 172.18.0.100
    Client-Ethernet-Address 00:00:00:11:22:33 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: ACK
      Server-ID Option 54, length 4: 05e1a4a4476e
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:18.869813 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:34 (oui Ethernet), length 251, xid 0x31c40ac1, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:34 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:18.872189 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:34 (oui Ethernet), length 263, xid 0x31c40ac1, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:34 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.101
      Server-ID Option 54, length 4: kea_1.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:22.559775 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:35 (oui Ethernet), length 251, xid 0x4c120d1c, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:35 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:22.561696 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    05e1a4a4476e.bootps > 172.18.0.102.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x4c120d1c, Flags [none]
    Your-IP 172.18.0.102
    Client-Ethernet-Address 00:00:00:11:22:35 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: Offer
      Server-ID Option 54, length 4: 05e1a4a4476e
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:38.856100 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:40 (oui Ethernet), length 251, xid 0x16e71cb, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:40 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:38.857816 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:40 (oui Ethernet), length 263, xid 0x16e71cb, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:40 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.107
      Server-ID Option 54, length 4: kea_1.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:41.995354 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:41 (oui Ethernet), length 251, xid 0x44f61b6e, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:41 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:41.996990 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:41 (oui Ethernet), length 263, xid 0x44f61b6e, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:41 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.108
      Server-ID Option 54, length 4: kea_1.kea-mng
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:46.075816 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:42 (oui Ethernet), length 251, xid 0x4da855db, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:42 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:46.077736 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    05e1a4a4476e.bootps > 172.18.0.109.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x4da855db, Flags [none]
    Your-IP 172.18.0.109
    Client-Ethernet-Address 00:00:00:11:22:42 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: Offer
      Server-ID Option 54, length 4: 05e1a4a4476e
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:46.078692 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:42 (oui Ethernet), length 263, xid 0x4da855db, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:42 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.109
      Server-ID Option 54, length 4: 05e1a4a4476e
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:47.091865 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    05e1a4a4476e.bootps > 172.18.0.109.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x4da855db, Flags [none]
    Your-IP 172.18.0.109
    Client-Ethernet-Address 00:00:00:11:22:42 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: ACK
      Server-ID Option 54, length 4: 05e1a4a4476e
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:49.416473 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:43 (oui Ethernet), length 251, xid 0x39e5fa04, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:43 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:49.418437 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    05e1a4a4476e.bootps > 172.18.0.110.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x39e5fa04, Flags [none]
    Your-IP 172.18.0.110
    Client-Ethernet-Address 00:00:00:11:22:43 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: Offer
      Server-ID Option 54, length 4: 05e1a4a4476e
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
06:59:49.418845 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:11:22:43 (oui Ethernet), length 263, xid 0x39e5fa04, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:43 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.0.110
      Server-ID Option 54, length 4: 05e1a4a4476e
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:59:50.424369 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    05e1a4a4476e.bootps > 172.18.0.110.bootpc: BOOTP/DHCP, Reply, length 290, xid 0x39e5fa04, Flags [none]
    Your-IP 172.18.0.110
    Client-Ethernet-Address 00:00:00:11:22:43 (oui Ethernet)
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      Subnet-Mask Option 1, length 4: 255.255.0.0
      Default-Gateway Option 3, length 4: 172.18.0.1
      Domain-Name-Server Option 6, length 8: dns.google,dns.google
      Lease-Time Option 51, length 4: 3600
      DHCP-Message Option 53, length 1: ACK
      Server-ID Option 54, length 4: 05e1a4a4476e
      RN Option 58, length 4: 1200
      RB Option 59, length 4: 2400
^C
20 packets captured
34 packets received by filter
14 packets dropped by kernel
root@05e1a4a4476e:/#

これら2つのパケットキャプチャをみて理解できるとおり， kea_1でもkea_2でも，はクライアントからのDHCP要求(DISCOVERY, REQUEST)において，クライアントに対してOFFER, ACK を返していることがわかる．これはHAモードのうち，両方のサーバがDHCPに応答するload-balancingの動作となっていることがわかる．ただし，kea_1とkea_2が完全に1回ごとに交互にリース動作をするのではなく，複数回同じserverが連続して応答することもあることが見て取れる．これはKea DHCPのload-balancingがRFC3074に基づいて実装されているからであるように見受けられる． RFC3074によると，DHCPDISCOVERパケットのある値からハッシュを取り，その値は0-255の値をとる．ロードバランシング構成がとられたそれぞれのサーバではあるテーブルをもつ．それは自分が応答するべき値が1となるように構成された32bytesのテーブルである(32 * 8 = 256)．この256ビットのテーブルと，DHCPDISCOVERパケットから計算されたハッシュ値を比較し，自分が応答すべき場合にのみDHCPOFFERパケットを送付して応答するロジックとなっているようだ．そのため，マクロ的，長期的にみると設計された負荷分散率となるが，ミクロ的，短時間的にみると偏りが生まれる可能性は十分に考えられる．今回繰り返して同一のサーバが応答したこともこの特徴的なアルゴリズムの振る舞いによるものだと考えることができるだろう．

追記: RFC3074を和訳しました．RFC 3074を読んでみた話 - JP7FKFの備忘録
ref: Kea High Availability vs ISC DHCP Failover - Kea DHCP

またこの時のlease-updateを見てみよう．

## kea_1

06:59:13.558097 IP (tos 0x0, ttl 64, id 5427, offset 0, flags [DF], proto TCP (6), length 405)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0x1fc4), seq 124:477, ack 255, win 321, options [nop,nop,TS val 7878270 ecr 7878004], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236753, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:33", "ip-address": "172.18.0.100", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:13.568131 IP (tos 0x0, ttl 64, id 176, offset 0, flags [DF], proto TCP (6), length 208)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x5eef), seq 255:411, ack 477, win 235, options [nop,nop,TS val 7878271 ecr 7878270], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:13 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:13.568225 IP (tos 0x0, ttl 64, id 5428, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x2781), ack 411, win 329, options [nop,nop,TS val 7878271 ecr 7878271], length 0
06:59:17.049416 IP (tos 0x0, ttl 64, id 24059, offset 0, flags [DF], proto TCP (6), length 176)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x7191), seq 124:248, ack 255, win 312, options [nop,nop,TS val 7878620 ecr 7877403], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:17.053342 IP (tos 0x0, ttl 64, id 6822, offset 0, flags [DF], proto TCP (6), length 306)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x594e (incorrect -> 0xdb0c), seq 255:509, ack 248, win 227, options [nop,nop,TS val 7878620 ecr 7878620], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:17 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:17 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:17.053369 IP (tos 0x0, ttl 64, id 24060, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xf4b4), ack 509, win 321, options [nop,nop,TS val 7878620 ecr 7878620], length 0
06:59:18.874313 IP (tos 0x0, ttl 64, id 24061, offset 0, flags [DF], proto TCP (6), length 405)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xe230), seq 248:601, ack 509, win 321, options [nop,nop,TS val 7878802 ecr 7878620], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236758, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:34", "ip-address": "172.18.0.101", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:18.877715 IP (tos 0x0, ttl 64, id 6823, offset 0, flags [DF], proto TCP (6), length 208)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x28ac), seq 509:665, ack 601, win 235, options [nop,nop,TS val 7878802 ecr 7878802], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:18 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:18.877843 IP (tos 0x0, ttl 64, id 24062, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xf143), ack 665, win 329, options [nop,nop,TS val 7878802 ecr 7878802], length 0
06:59:22.561975 IP (tos 0x0, ttl 64, id 5429, offset 0, flags [DF], proto TCP (6), length 176)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x9a95), seq 477:601, ack 411, win 329, options [nop,nop,TS val 7879171 ecr 7878271], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:22.563407 IP (tos 0x0, ttl 64, id 177, offset 0, flags [DF], proto TCP (6), length 306)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x594e (incorrect -> 0x0f55), seq 411:665, ack 601, win 235, options [nop,nop,TS val 7879171 ecr 7879171], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:22 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:22 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:22.563509 IP (tos 0x0, ttl 64, id 5430, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x1ef7), ack 665, win 337, options [nop,nop,TS val 7879171 ecr 7879171], length 0
06:59:22.574735 IP (tos 0x0, ttl 64, id 5431, offset 0, flags [DF], proto TCP (6), length 405)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0x1127), seq 601:954, ack 665, win 337, options [nop,nop,TS val 7879172 ecr 7879171], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236762, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:35", "ip-address": "172.18.0.102", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:22.584285 IP (tos 0x0, ttl 64, id 178, offset 0, flags [DF], proto TCP (6), length 208)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x5365), seq 665:821, ack 954, win 243, options [nop,nop,TS val 7879173 ecr 7879172], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:22 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:22.628444 IP (tos 0x0, ttl 64, id 5432, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x1ce8), ack 821, win 346, options [nop,nop,TS val 7879178 ecr 7879173], length 0
06:59:26.274709 IP (tos 0x0, ttl 64, id 5433, offset 0, flags [DF], proto TCP (6), length 405)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0x07ad), seq 954:1307, ack 821, win 346, options [nop,nop,TS val 7879542 ecr 7879173], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236766, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:36", "ip-address": "172.18.0.103", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:26.327869 IP (tos 0x0, ttl 64, id 179, offset 0, flags [DF], proto TCP (6), length 208)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x4e73), seq 821:977, ack 1307, win 252, options [nop,nop,TS val 7879547 ecr 7879542], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:26 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:26.328036 IP (tos 0x0, ttl 64, id 5434, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x17fc), ack 977, win 354, options [nop,nop,TS val 7879547 ecr 7879547], length 0
06:59:28.296754 IP (tos 0x0, ttl 64, id 24063, offset 0, flags [DF], proto TCP (6), length 176)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x642a), seq 601:725, ack 665, win 329, options [nop,nop,TS val 7879748 ecr 7878802], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:28.298602 IP (tos 0x0, ttl 64, id 6824, offset 0, flags [DF], proto TCP (6), length 306)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x594e (incorrect -> 0xccbb), seq 665:919, ack 725, win 235, options [nop,nop,TS val 7879748 ecr 7879748], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:28 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:28 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:28.298650 IP (tos 0x0, ttl 64, id 24064, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xe85d), ack 919, win 337, options [nop,nop,TS val 7879748 ecr 7879748], length 0
06:59:29.229170 IP (tos 0x0, ttl 64, id 5435, offset 0, flags [DF], proto TCP (6), length 405)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xfe06), seq 1307:1660, ack 977, win 354, options [nop,nop,TS val 7879841 ecr 7879547], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236769, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:37", "ip-address": "172.18.0.104", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:29.230833 IP (tos 0x0, ttl 64, id 180, offset 0, flags [DF], proto TCP (6), length 208)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x4a1a), seq 977:1133, ack 1660, win 260, options [nop,nop,TS val 7879841 ecr 7879841], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:29 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:29.230925 IP (tos 0x0, ttl 64, id 5436, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x13aa), ack 1133, win 363, options [nop,nop,TS val 7879841 ecr 7879841], length 0
06:59:32.314046 IP (tos 0x0, ttl 64, id 24065, offset 0, flags [DF], proto TCP (6), length 405)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xd2fc), seq 725:1078, ack 919, win 337, options [nop,nop,TS val 7880149 ecr 7879748], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236772, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:38", "ip-address": "172.18.0.105", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:32.317924 IP (tos 0x0, ttl 64, id 6825, offset 0, flags [DF], proto TCP (6), length 208)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x18ac), seq 919:1075, ack 1078, win 243, options [nop,nop,TS val 7880150 ecr 7880149], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:32 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:32.317973 IP (tos 0x0, ttl 64, id 24066, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xe333), ack 1075, win 346, options [nop,nop,TS val 7880150 ecr 7880150], length 0
06:59:33.319217 IP (tos 0x0, ttl 64, id 5437, offset 0, flags [DF], proto TCP (6), length 176)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x88a9), seq 1660:1784, ack 1133, win 363, options [nop,nop,TS val 7880250 ecr 7879841], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:33.322421 IP (tos 0x0, ttl 64, id 181, offset 0, flags [DF], proto TCP (6), length 306)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x594e (incorrect -> 0xfd5a), seq 1133:1387, ack 1784, win 260, options [nop,nop,TS val 7880250 ecr 7880250], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:33 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:33 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:33.322543 IP (tos 0x0, ttl 64, id 5438, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x0ef6), ack 1387, win 371, options [nop,nop,TS val 7880250 ecr 7880250], length 0
06:59:35.557413 IP (tos 0x0, ttl 64, id 24067, offset 0, flags [DF], proto TCP (6), length 405)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xc91f), seq 1078:1431, ack 1075, win 346, options [nop,nop,TS val 7880474 ecr 7880150], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236775, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:39", "ip-address": "172.18.0.106", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:35.559028 IP (tos 0x0, ttl 64, id 6826, offset 0, flags [DF], proto TCP (6), length 208)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x141a), seq 1075:1231, ack 1431, win 252, options [nop,nop,TS val 7880474 ecr 7880474], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:35 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:35.559069 IP (tos 0x0, ttl 64, id 24068, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xdea6), ack 1231, win 354, options [nop,nop,TS val 7880474 ecr 7880474], length 0
06:59:38.859078 IP (tos 0x0, ttl 64, id 24069, offset 0, flags [DF], proto TCP (6), length 405)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xc98b), seq 1431:1784, ack 1231, win 354, options [nop,nop,TS val 7880804 ecr 7880474], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236778, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:40", "ip-address": "172.18.0.107", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:38.861084 IP (tos 0x0, ttl 64, id 6827, offset 0, flags [DF], proto TCP (6), length 208)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x0f7e), seq 1231:1387, ack 1784, win 260, options [nop,nop,TS val 7880804 ecr 7880804], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:38 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:38.861128 IP (tos 0x0, ttl 64, id 24070, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xda0c), ack 1387, win 363, options [nop,nop,TS val 7880804 ecr 7880804], length 0
06:59:39.860222 IP (tos 0x0, ttl 64, id 24071, offset 0, flags [DF], proto TCP (6), length 176)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x5041), seq 1784:1908, ack 1387, win 363, options [nop,nop,TS val 7880904 ecr 7880804], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:39.864235 IP (tos 0x0, ttl 64, id 6828, offset 0, flags [DF], proto TCP (6), length 306)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x594e (incorrect -> 0xba27), seq 1387:1641, ack 1908, win 260, options [nop,nop,TS val 7880904 ecr 7880904], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:39 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:39 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:39.864285 IP (tos 0x0, ttl 64, id 24072, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xd7c1), ack 1641, win 371, options [nop,nop,TS val 7880905 ecr 7880904], length 0
06:59:41.998380 IP (tos 0x0, ttl 64, id 24073, offset 0, flags [DF], proto TCP (6), length 405)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xc81a), seq 1908:2261, ack 1641, win 371, options [nop,nop,TS val 7881118 ecr 7880904], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236781, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:41", "ip-address": "172.18.0.108", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:42.008824 IP (tos 0x0, ttl 64, id 6829, offset 0, flags [DF], proto TCP (6), length 208)
    kea_2.kea-mng.http-alt > 085fd121c5c6.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x088f), seq 1641:1797, ack 2261, win 269, options [nop,nop,TS val 7881119 ecr 7881118], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:42 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:42.008877 IP (tos 0x0, ttl 64, id 24074, offset 0, flags [DF], proto TCP (6), length 52)
    085fd121c5c6.44216 > kea_2.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xd40f), ack 1797, win 379, options [nop,nop,TS val 7881119 ecr 7881119], length 0
06:59:45.015705 IP (tos 0x0, ttl 64, id 5439, offset 0, flags [DF], proto TCP (6), length 176)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x80fc), seq 1784:1908, ack 1387, win 371, options [nop,nop,TS val 7881420 ecr 7880250], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:45.022784 IP (tos 0x0, ttl 64, id 182, offset 0, flags [DF], proto TCP (6), length 306)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x594e (incorrect -> 0xeeba), seq 1387:1641, ack 1908, win 260, options [nop,nop,TS val 7881420 ecr 7881420], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:45 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:45 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:45.022912 IP (tos 0x0, ttl 64, id 5440, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x0450), ack 1641, win 379, options [nop,nop,TS val 7881420 ecr 7881420], length 0
06:59:46.089643 IP (tos 0x0, ttl 64, id 5441, offset 0, flags [DF], proto TCP (6), length 405)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xee12), seq 1908:2261, ack 1641, win 379, options [nop,nop,TS val 7881527 ecr 7881420], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236786, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:42", "ip-address": "172.18.0.109", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:46.100810 IP (tos 0x0, ttl 64, id 183, offset 0, flags [DF], proto TCP (6), length 208)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x35f6), seq 1641:1797, ack 2261, win 269, options [nop,nop,TS val 7881528 ecr 7881527], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:46 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:46.101035 IP (tos 0x0, ttl 64, id 5442, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x0172), ack 1797, win 388, options [nop,nop,TS val 7881528 ecr 7881528], length 0
06:59:49.422866 IP (tos 0x0, ttl 64, id 5443, offset 0, flags [DF], proto TCP (6), length 405)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xef52), seq 2261:2614, ack 1797, win 388, options [nop,nop,TS val 7881860 ecr 7881528], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236789, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:43", "ip-address": "172.18.0.110", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:49.430796 IP (tos 0x0, ttl 64, id 184, offset 0, flags [DF], proto TCP (6), length 208)
    085fd121c5c6.http-alt > kea_2.kea-mng.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x3154), seq 1797:1953, ack 2614, win 277, options [nop,nop,TS val 7881861 ecr 7881860], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:49 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:49.430907 IP (tos 0x0, ttl 64, id 5444, offset 0, flags [DF], proto TCP (6), length 52)
    kea_2.kea-mng.47044 > 085fd121c5c6.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xfcd2), ack 1953, win 396, options [nop,nop,TS val 7881861 ecr 7881861], length 0

## kea_2

06:59:13.558062 IP (tos 0x0, ttl 64, id 5427, offset 0, flags [DF], proto TCP (6), length 405)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0x1fc4), seq 124:477, ack 255, win 321, options [nop,nop,TS val 7878270 ecr 7878004], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236753, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:33", "ip-address": "172.18.0.100", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:13.568183 IP (tos 0x0, ttl 64, id 176, offset 0, flags [DF], proto TCP (6), length 208)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x5eef), seq 255:411, ack 477, win 235, options [nop,nop,TS val 7878271 ecr 7878270], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:13 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:13.568211 IP (tos 0x0, ttl 64, id 5428, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x2781), ack 411, win 329, options [nop,nop,TS val 7878271 ecr 7878271], length 0
06:59:17.049679 IP (tos 0x0, ttl 64, id 24059, offset 0, flags [DF], proto TCP (6), length 176)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x7191), seq 124:248, ack 255, win 312, options [nop,nop,TS val 7878620 ecr 7877403], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:17.053316 IP (tos 0x0, ttl 64, id 6822, offset 0, flags [DF], proto TCP (6), length 306)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x594e (incorrect -> 0xdb0c), seq 255:509, ack 248, win 227, options [nop,nop,TS val 7878620 ecr 7878620], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:17 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:17 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:17.053384 IP (tos 0x0, ttl 64, id 24060, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xf4b4), ack 509, win 321, options [nop,nop,TS val 7878620 ecr 7878620], length 0
06:59:18.874376 IP (tos 0x0, ttl 64, id 24061, offset 0, flags [DF], proto TCP (6), length 405)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xe230), seq 248:601, ack 509, win 321, options [nop,nop,TS val 7878802 ecr 7878620], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236758, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:34", "ip-address": "172.18.0.101", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:18.877454 IP (tos 0x0, ttl 64, id 6823, offset 0, flags [DF], proto TCP (6), length 208)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x28ac), seq 509:665, ack 601, win 235, options [nop,nop,TS val 7878802 ecr 7878802], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:18 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:18.877877 IP (tos 0x0, ttl 64, id 24062, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xf143), ack 665, win 329, options [nop,nop,TS val 7878802 ecr 7878802], length 0
06:59:22.561932 IP (tos 0x0, ttl 64, id 5429, offset 0, flags [DF], proto TCP (6), length 176)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x9a95), seq 477:601, ack 411, win 329, options [nop,nop,TS val 7879171 ecr 7878271], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:22.563444 IP (tos 0x0, ttl 64, id 177, offset 0, flags [DF], proto TCP (6), length 306)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x594e (incorrect -> 0x0f55), seq 411:665, ack 601, win 235, options [nop,nop,TS val 7879171 ecr 7879171], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:22 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:22 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:22.563492 IP (tos 0x0, ttl 64, id 5430, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x1ef7), ack 665, win 337, options [nop,nop,TS val 7879171 ecr 7879171], length 0
06:59:22.574582 IP (tos 0x0, ttl 64, id 5431, offset 0, flags [DF], proto TCP (6), length 405)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0x1127), seq 601:954, ack 665, win 337, options [nop,nop,TS val 7879172 ecr 7879171], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236762, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:35", "ip-address": "172.18.0.102", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:22.584353 IP (tos 0x0, ttl 64, id 178, offset 0, flags [DF], proto TCP (6), length 208)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x5365), seq 665:821, ack 954, win 243, options [nop,nop,TS val 7879173 ecr 7879172], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:22 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:22.628326 IP (tos 0x0, ttl 64, id 5432, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x1ce8), ack 821, win 346, options [nop,nop,TS val 7879178 ecr 7879173], length 0
06:59:26.274675 IP (tos 0x0, ttl 64, id 5433, offset 0, flags [DF], proto TCP (6), length 405)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0x07ad), seq 954:1307, ack 821, win 346, options [nop,nop,TS val 7879542 ecr 7879173], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236766, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:36", "ip-address": "172.18.0.103", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:26.327969 IP (tos 0x0, ttl 64, id 179, offset 0, flags [DF], proto TCP (6), length 208)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x4e73), seq 821:977, ack 1307, win 252, options [nop,nop,TS val 7879547 ecr 7879542], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:26 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:26.328020 IP (tos 0x0, ttl 64, id 5434, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x17fc), ack 977, win 354, options [nop,nop,TS val 7879547 ecr 7879547], length 0
06:59:28.296804 IP (tos 0x0, ttl 64, id 24063, offset 0, flags [DF], proto TCP (6), length 176)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x642a), seq 601:725, ack 665, win 329, options [nop,nop,TS val 7879748 ecr 7878802], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:28.298553 IP (tos 0x0, ttl 64, id 6824, offset 0, flags [DF], proto TCP (6), length 306)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x594e (incorrect -> 0xccbb), seq 665:919, ack 725, win 235, options [nop,nop,TS val 7879748 ecr 7879748], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:28 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:28 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:28.298689 IP (tos 0x0, ttl 64, id 24064, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xe85d), ack 919, win 337, options [nop,nop,TS val 7879748 ecr 7879748], length 0
06:59:29.229138 IP (tos 0x0, ttl 64, id 5435, offset 0, flags [DF], proto TCP (6), length 405)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xfe06), seq 1307:1660, ack 977, win 354, options [nop,nop,TS val 7879841 ecr 7879547], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236769, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:37", "ip-address": "172.18.0.104", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:29.230883 IP (tos 0x0, ttl 64, id 180, offset 0, flags [DF], proto TCP (6), length 208)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x4a1a), seq 977:1133, ack 1660, win 260, options [nop,nop,TS val 7879841 ecr 7879841], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:29 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:29.230912 IP (tos 0x0, ttl 64, id 5436, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x13aa), ack 1133, win 363, options [nop,nop,TS val 7879841 ecr 7879841], length 0
06:59:32.314326 IP (tos 0x0, ttl 64, id 24065, offset 0, flags [DF], proto TCP (6), length 405)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xd2fc), seq 725:1078, ack 919, win 337, options [nop,nop,TS val 7880149 ecr 7879748], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236772, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:38", "ip-address": "172.18.0.105", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:32.317838 IP (tos 0x0, ttl 64, id 6825, offset 0, flags [DF], proto TCP (6), length 208)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x18ac), seq 919:1075, ack 1078, win 243, options [nop,nop,TS val 7880150 ecr 7880149], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:32 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:32.317988 IP (tos 0x0, ttl 64, id 24066, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xe333), ack 1075, win 346, options [nop,nop,TS val 7880150 ecr 7880150], length 0
06:59:33.319178 IP (tos 0x0, ttl 64, id 5437, offset 0, flags [DF], proto TCP (6), length 176)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x88a9), seq 1660:1784, ack 1133, win 363, options [nop,nop,TS val 7880250 ecr 7879841], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:33.322475 IP (tos 0x0, ttl 64, id 181, offset 0, flags [DF], proto TCP (6), length 306)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x594e (incorrect -> 0xfd5a), seq 1133:1387, ack 1784, win 260, options [nop,nop,TS val 7880250 ecr 7880250], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:33 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:33 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:33.322526 IP (tos 0x0, ttl 64, id 5438, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x0ef6), ack 1387, win 371, options [nop,nop,TS val 7880250 ecr 7880250], length 0
06:59:35.557433 IP (tos 0x0, ttl 64, id 24067, offset 0, flags [DF], proto TCP (6), length 405)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xc91f), seq 1078:1431, ack 1075, win 346, options [nop,nop,TS val 7880474 ecr 7880150], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236775, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:39", "ip-address": "172.18.0.106", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:35.558987 IP (tos 0x0, ttl 64, id 6826, offset 0, flags [DF], proto TCP (6), length 208)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x141a), seq 1075:1231, ack 1431, win 252, options [nop,nop,TS val 7880474 ecr 7880474], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:35 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:35.559082 IP (tos 0x0, ttl 64, id 24068, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xdea6), ack 1231, win 354, options [nop,nop,TS val 7880474 ecr 7880474], length 0
06:59:38.859100 IP (tos 0x0, ttl 64, id 24069, offset 0, flags [DF], proto TCP (6), length 405)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xc98b), seq 1431:1784, ack 1231, win 354, options [nop,nop,TS val 7880804 ecr 7880474], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236778, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:40", "ip-address": "172.18.0.107", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:38.861043 IP (tos 0x0, ttl 64, id 6827, offset 0, flags [DF], proto TCP (6), length 208)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x0f7e), seq 1231:1387, ack 1784, win 260, options [nop,nop,TS val 7880804 ecr 7880804], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:38 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:38.861141 IP (tos 0x0, ttl 64, id 24070, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xda0c), ack 1387, win 363, options [nop,nop,TS val 7880804 ecr 7880804], length 0
06:59:39.860249 IP (tos 0x0, ttl 64, id 24071, offset 0, flags [DF], proto TCP (6), length 176)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x5041), seq 1784:1908, ack 1387, win 363, options [nop,nop,TS val 7880904 ecr 7880804], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:39.864068 IP (tos 0x0, ttl 64, id 6828, offset 0, flags [DF], proto TCP (6), length 306)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x594e (incorrect -> 0xba27), seq 1387:1641, ack 1908, win 260, options [nop,nop,TS val 7880904 ecr 7880904], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:39 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:39 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:39.864299 IP (tos 0x0, ttl 64, id 24072, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xd7c1), ack 1641, win 371, options [nop,nop,TS val 7880905 ecr 7880904], length 0
06:59:41.998412 IP (tos 0x0, ttl 64, id 24073, offset 0, flags [DF], proto TCP (6), length 405)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xc81a), seq 1908:2261, ack 1641, win 371, options [nop,nop,TS val 7881118 ecr 7880904], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236781, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:41", "ip-address": "172.18.0.108", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:42.008785 IP (tos 0x0, ttl 64, id 6829, offset 0, flags [DF], proto TCP (6), length 208)
    05e1a4a4476e.http-alt > kea_1.kea-mng.44216: Flags [P.], cksum 0x58ec (incorrect -> 0x088f), seq 1641:1797, ack 2261, win 269, options [nop,nop,TS val 7881119 ecr 7881118], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:42 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:42.008894 IP (tos 0x0, ttl 64, id 24074, offset 0, flags [DF], proto TCP (6), length 52)
    kea_1.kea-mng.44216 > 05e1a4a4476e.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0xd40f), ack 1797, win 379, options [nop,nop,TS val 7881119 ecr 7881119], length 0
06:59:45.015655 IP (tos 0x0, ttl 64, id 5439, offset 0, flags [DF], proto TCP (6), length 176)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x58cc (incorrect -> 0x80fc), seq 1784:1908, ack 1387, win 371, options [nop,nop,TS val 7881420 ecr 7880250], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:59:45.022860 IP (tos 0x0, ttl 64, id 182, offset 0, flags [DF], proto TCP (6), length 306)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x594e (incorrect -> 0xeeba), seq 1387:1641, ack 1908, win 260, options [nop,nop,TS val 7881420 ecr 7881420], length 254: HTTP, length: 254
  HTTP/1.1 200 OK
  Content-Length: 145
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:45 GMT

  [ { "arguments": { "date-time": "Wed, 20 Nov 2019 06:59:45 GMT", "state": "load-balancing" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:59:45.022897 IP (tos 0x0, ttl 64, id 5440, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x0450), ack 1641, win 379, options [nop,nop,TS val 7881420 ecr 7881420], length 0
06:59:46.089589 IP (tos 0x0, ttl 64, id 5441, offset 0, flags [DF], proto TCP (6), length 405)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xee12), seq 1908:2261, ack 1641, win 379, options [nop,nop,TS val 7881527 ecr 7881420], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236786, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:42", "ip-address": "172.18.0.109", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:46.100884 IP (tos 0x0, ttl 64, id 183, offset 0, flags [DF], proto TCP (6), length 208)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x35f6), seq 1641:1797, ack 2261, win 269, options [nop,nop,TS val 7881528 ecr 7881527], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:46 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:59:46.100966 IP (tos 0x0, ttl 64, id 5442, offset 0, flags [DF], proto TCP (6), length 52)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [.], cksum 0x5850 (incorrect -> 0x0172), ack 1797, win 388, options [nop,nop,TS val 7881528 ecr 7881528], length 0
06:59:49.422817 IP (tos 0x0, ttl 64, id 5443, offset 0, flags [DF], proto TCP (6), length 405)
    05e1a4a4476e.47044 > kea_1.kea-mng.http-alt: Flags [P.], cksum 0x59b1 (incorrect -> 0xef52), seq 2261:2614, ack 1797, win 388, options [nop,nop,TS val 7881860 ecr 7881528], length 353: HTTP, length: 353
  POST / HTTP/1.1
  Content-Length: 281
  Content-Type: application/json

  { "arguments": { "expire": 1574236789, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:43", "ip-address": "172.18.0.110", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:59:49.430854 IP (tos 0x0, ttl 64, id 184, offset 0, flags [DF], proto TCP (6), length 208)
    kea_1.kea-mng.http-alt > 05e1a4a4476e.47044: Flags [P.], cksum 0x58ec (incorrect -> 0x3154), seq 1797:1953, ack 2614, win 277, options [nop,nop,TS val 7881861 ecr 7881860], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Wed, 20 Nov 2019 06:59:49 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]

上記より，kea_1とkea_2との間で双方向にリースアップデートを更新していることがことがわかる． load-balancingモードの場合は，hot-standbyモードとは異なり，リースしたサーバからそうでないサーバ（群）に対してリースアップデートを行うため，リースアップデートが一方向のみでなく，双方向に行われる違いがある．これがパフォーマンス低下の原因となるため，一般にhot-standbyモードよりもload-balancingモードの方が応答が遅くなる傾向があるようである．

ref: Kea Performance Optimization section 7の Consider the impact of high availability に
Active-Passive is more efficient than load-balancing. をはじめとし，load-balancingモードとhot-standbyモードでのパフォーマンスの違いの解説が書かれている．

まとめ

Kea DHCPを使ってload-balancingモードのHA構成を組むことができました．本記事では以下のことを実施しました．

ISCのKea DHCPを用いてload-balancingモードのHA構成を組むことについて紹介した．
load-balancingモードのHAにおいて実際にクライアントを模擬してDHCPによるアドレスリースを行い，hot-standbyモードと比べたの応答の違いをパケットキャプチャによって確認した．

次回以降では，RFC3074の掘り下げ，HA構成を取ることによるパフォーマンス影響，HA構成で本当に冗長化ができているのか，等について見てみようと考えています．

References

P.S.

例によってQiitaに投稿してみました
- Kea DHCPでAct-Act(load-balancing)のHA構成を組む - Qiita

2019-10-31

Kea DHCPのHigh-Availabilityを検証する - (1)Act-Sbyのhot-standby構成を組む

network software

ゴール

Kea DHCPを用いてAct-Sbyのhot-standby High-Availability構成を組み，動作させることができること．

About Kea DHCP

Kea DHCPとは，ISC DHCPの後継として開発されているオープンソースのDHCPサーバプログラムである． MPL2.0ライセンスのもとISCのGitlabで公開されている．最新のメジャーバージョンはISC DHCPとの違いは下記のように書かれている．

1. モジュラーコンポーネントデザインとHooksモジュールによる拡張
2. REST APIによるオンラインRe-configuration
3. 既存のシステムとのインテグレーション可能なデザイン

1つ目に述べられているhooksモジュールについては，ユーザ自身でこのhooksモジュールを書くことが可能であることを示している．これによりKea DHCPの独自のモジュールを実装し，より柔軟なサービス提供ができるように考慮されている．今回対象とするHA構成についても，このhooksモジュールとして実装されているHA機能を提供するライブラリをincludeして利用する形で実現する．

2つ目に述べられているREST APIによるconfigurationについては，このKea DHCPにはkea-ctrl-agentが内包されており，このagentがREST APIの仲介者となりkea-dhcp4プロセス(DHCPv4)やkea-dhcp6プロセス(DHCPv6)に対して各種設定のgetやset等の様々な操作をすることが可能である．またKea DHCPはconfigを変更した際にrestartが極力必要とならないよう設計されている．多くのconfiguratinにおいては，configのreloadを行うことでconfigの適用を行うことができる．

最後に，既存システムとのインテグレーションについて述べられているが，これはどうもKea DHCPのconfigurationを，configファイルだけでなくDatabaseを用いることでも行うことができる点を指しているようである．これにより，実行環境とconfigurationを分離することができることがこの3つ目の要旨であるようだ．たしかに，実行環境とconfigの環境が異なり，さらにDBを用いて管理をすることができるのは一部のIT管理者にとっては有効かもしれない．

Kea DHCPのHA構成とISC DHCPのFailover

かつてのISC-DHCPではHA機能は提供されておらず，Failover機能のみ提供されているものであった．しかしながらKea DHCPではrelease 1.4.0よりHA構成が適用可能となった．リリースノートからも分かる通り，1.4.0は2018年6月にリリースされたものであり，まだ1年少々しか経過していない．そのためまだ機能が十分でない部分等も多くあると推察できる．1.4.0のリリース当初はこのHA機能はPremium featureとして提供されるようであったが，現在は通常のhooks libraryとして提供されている．

ISC-DHCPのFailoverとKea DHCPのHA構成については下記で述べられている．
Kea High Availability vs ISC DHCP Failover - Kea DHCP
比較してみると，ISCのFailoverがDHCPv4のみに適用可能だったのに対し，Keaではv4とv6の両方に対して適用可能となっている．他には，ISCでは2つのサーバでの冗長構成のみしか組むことができなかったが，Keaでは2つのactiveサーバと無限のバックアップサーバを用いることが可能である．他にもリースDBのレプリケーション等がKeaでは実施可能であったり，ISC DHCPではOMAPIであったが，Keaではより一般的となっているREST APIによるconfigurationを採用するなど，多くの点でKea DHCPはISC DHCPをfollowする形となっているが，一部機能においてはISCよりデグレしている点が見受けられる．たとえばLoadBalancingだと，ISCではFlexible LBが可能であったが，Keaでは50/50のLBモードしか現状選択することができない．いずれもRFC3074に準拠しているようだ．また，Lazy lease updates(MCLT)の項目についてはISC DHCPでは可能であったがKeaではこれを許さず，リースアップデートが完全に完了してからでなければclientへresponseを送らないなど，より厳密となっている．このLazy lease updates(MCLT)については特にHA構成を組んだ場合のパフォーマンスに大きく影響することとなる．

Kea DHCPでHA構成を組む

Introduction

では早速Kea DHCPを用いたHA構成を組んでいく．今回はKea DHCP verion 1.5.0を利用する．基本的にはマニュアルを熟読の上設定を行なっていくこととなる．v1.5.0では下記を参照することとなるだろう．
Kea Administrator Reference Manual
HA構成を組む上ではこのマニュアルのうち 15.4.8. ha: High Availability をrefer するとよい．また，HA構成において，HA構成を組んだホスト間のリースアップデート等はkea-ctrl-agentを介したREST API経由で実施されることとなる．そのため，Kea DHCPのREST API，kea-ctrl-agentについての理解も深めておくとより理解が深まると思う．

Build and Configuration

今回はdockerを用いて検証環境を構成してHA構成を組むことを試みる．
下記のdocker file とdocker-compose.ymlを用いてkea-dhcp2台をデプロイし，その間をkea-mngネットワークで繋いでおく．docker compose upする前に，docker network create kea-mng をしておく必要がある．これはHA構成を組むにあたりheartbeatやlease-updateをREST APIで実施するためのmanagement network相当のものである．L3疎通性さえあれば問題はない．

FROM ubuntu:latest

ENV KEA_DHCP_VERSION=1.5.0
ENV LOG4_CPLUS_VERSION=1.2.1

RUN apt-get update && apt-get install -y iputils-ping net-tools traceroute nmap tcpdump iproute2 python3 vim  jq curl automake libtool pkg-config build-essential ccache libboost-dev libboost-system-dev liblog4cplus-dev libssl-dev && \
    curl -SL https://downloads.isc.org/isc/kea/${KEA_DHCP_VERSION}/kea-${KEA_DHCP_VERSION}.tar.gz | tar -xzC /root && \
    cd /root/kea-${KEA_DHCP_VERSION} && \
    ./configure --enable-shell && \
    make -s -j$(nproc) && \
    make install && \
    ldconfig

version: '3'
services:
  kea_1:
    image: kea:1.5.0
    container_name: kea_1
    privileged: true
    networks:
      - kea-mng
    tty: true

  kea_2:
    image: kea:1.5.0
    container_name: kea_2
    privileged: true
    networks:
      - kea-mng
    tty: true

networks:
    kea-mng:
        external: true

環境が構築できたら，実際にconfigを行っていく． manualを見て分かるとおり，HA構成の実現にはhooks libraryを用いる．今回はDHCPv4のデーモンに対してHA構成を行う例を示す．DHCPv6に対しても似たconfigで実現できると思われる．

早速HA構成を組むためのconfigを見ていこう．
マニュアルのHA構成のconfigサンプルとしてこのような例がある．

{
  "Dhcp4": {

    ...

    "hooks-libraries": [
        {
            "library": "/usr/lib/hooks/libdhcp_lease_cmds.so",
            "parameters": { }
        },
        {
            "library": "/usr/lib/hooks/libdhcp_ha.so",
            "parameters": {
                "high-availability": [ {
                    "this-server-name": "server1",
                    "mode": "load-balancing",
                    "heartbeat-delay": 10000,
                    "max-response-delay": 10000,
                    "max-ack-delay": 5000,
                    "max-unacked-clients": 5,
                    "peers": [
                        {
                            "name": "server1",
                            "url": "http://192.168.56.33:8080/",
                            "role": "primary",
                            "auto-failover": true
                        },
                        {
                            "name": "server2",
                            "url": "http://192.168.56.66:8080/",
                            "role": "secondary",
                            "auto-failover": true
                        },
                        {
                            "name": "server3",
                            "url": "http://192.168.56.99:8080/",
                            "role": "backup",
                            "auto-failover": false
                        }
                    ]
                } ]
            }
        }
    ],

    ...

  }
}

hooks-librariesとしてlibdhcp_ha.soを読み込む．デフォルトでは/user/lib/hooks配下に存在するが，keaのinstall pathによっては異なってくるので注意しよう．基本的にはkeaのroot配下の./lib/hooks配下にあると考えて良さそうだ．またHA構成を組む場合，lease-updateを送受信してリース情報を交換する必要がある．これを行うためにlibdhcp_lease_cmds.soを読み込む必要がある．読み込んでいないとlease情報がAPI経由でやり取りすることができない．たとえばこのlibdhcp_lease_cmd.soを読み込まずにlease情報をgetしようとすると，下記のようになる．

root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "lease4-get-all", "service": [ "dhcp4" ] }' http://localhost:8080/ | jq .[]
{
  "result": 2,
  "text": "'lease4-get-all' command not supported."
}

さらに各種パラメータのsettingもここで実施する．パラメータの意味はざっくりと下記の通りである．

this-server-name: HA構成のうち自身を指すユニークな名前．
mode: HAのモード．
heartbeat-delay: パートナーへのハートビートの送信間隔．[ms]
max-response-delay: パートナーをdownと判定するまでの時間．[ms]
max-ack-delay: パートナーがクライアントからの通信に応答するべきmax時間．これを超えるとunackedと判定される．[ms]
max-unacked-clients: パートナーを障害と判定するunackedクライアント数．
peers: HAを構成するホストの指定

これらをマニュアルを見つつ適切に設定することとなる．このうち，動作を大きく変えるのが mode である．このパラメータはHAをどのモードで動作させるかを決定する． Kea DHCPのHA動作モードはload-balancingとhot-standbyのうちから選ぶ．想像できる通り，load-balancingモードでは50/50のact-actによる構成，hot-standbyモードではact-sbyのホットスタンバイ構成となる．

今回はまずhot-standbyモードとして下記の通り設定することとする． primary に相当するのがkea_1でありIPアドレスは172.18.0.3，secondaryに相当するのがkea_2でありIPアドレスは172.18.0.2とした．

this-server-name: kea_1 //(2台目は"kea_2")
mode: hot-standby
heartbeat-delay: 10000
max-response-delay: 60000
max-ack-delay: 10000
max-unacked-clients: 0
"peers": [
    {
        "name": "kea_1",
        "url": "http://172.18.0.3:8080/",
        "role": "primary",
        "auto-failover": true
    },
    {
        "name": "kea_2",
        "url": "http://172.18.0.2:8080/",
        "role": "secondary",
        "auto-failover": true
    }
]

上記を踏まえてconfigを作成し，各サーバに適用して動作を確認してみよう．
たとえば私の検証では下記のようなconfgをkea_1に投入した．kea_2もほぼ同様であるが，"this-server-name" 等を適切に変更する．

{
"Dhcp4": {
    "interfaces-config": {
        "interfaces": ["eth0"],
        "dhcp-socket-type": "raw",
        "outbound-interface": "use-routing"
    },
    "control-socket": {
        "socket-type": "unix",
        "socket-name": "/tmp/kea-dhcp4-ctrl.sock"
    },
    "lease-database": {
        "type": "memfile",
        "persist": true,
        "name": "/usr/local/etc/kea/kea-leases4.csv",
        "lfc-interval": 1800
    },
    "expired-leases-processing": {
        "reclaim-timer-wait-time": 10,
        "flush-reclaimed-timer-wait-time": 25,
        "hold-reclaimed-time": 3600,
        "max-reclaim-leases": 100,
        "max-reclaim-time": 250,
        "unwarned-reclaim-cycles": 5
    },

    "renew-timer": 1200,
    "rebind-timer": 2400,
    "valid-lifetime": 3600,

    "option-data": [
        {
            "name": "domain-name-servers",
            "data": "8.8.8.8, 8.8.4.4"
        },
        {
            "name": "default-ip-ttl",
            "data": "0xf0"
        }
    ],

    "hooks-libraries": [
        {
            "library": "/usr/local/lib/hooks/libdhcp_lease_cmds.so",
            "parameters": { }
        },
        {
            "library": "/usr/local/lib/hooks/libdhcp_ha.so",
            "parameters": {
                "high-availability": [ {
                    "this-server-name": "kea_1",
                    "mode": "hot-standby",
                    "heartbeat-delay": 10000,
                    "max-response-delay": 10000,
                    "max-ack-delay": 5000,
                    "max-unacked-clients": 5,
                    "peers": [
                      {
                        "name": "kea_1",
                        "url": "http://172.18.0.2:8080/",
                        "role": "primary",
                        "auto-failover": true
                      },
                      {
                        "name": "kea_2",
                        "url": "http://172.18.0.3:8080/",
                        "role": "standby",
                        "auto-failover": true
                      }
                    ]
                } ]
            }
        }
    ],

    "reservation-mode": "disabled",
    "host-reservation-identifiers": [ "hw-address" ],

    "subnet4": [
          {
                "id": 1,
                "subnet": "172.18.0.0/16",
                "pools": [
                    { "pool": "172.18.1.1 - 172.18.1.250" }
                ],
                "option-data": [
                    {
                        "name": "routers",
                        "data": "172.18.0.1"
                    }
                ]
          }
    ]
},

"Logging":
{
  "loggers": [
    {
        "name": "kea-dhcp4",
        "output_options": [
            {
                "output": "/usr/local/var/log/kea-dhcp4.log",
                "maxver": 8,
                "maxsize": 204800,
                "flush": true
            }
        ],
        "severity": "DEBUG",
        "debuglevel": 99
    }
  ]
}
}

まずはkea_1でkeaを起動する．今回，起動にはkeactrlコマンドを用いた．起動後にRESTAPIでha-heartbeatを送信し，それぞれのサーバのステータスを見てみよう．

# kea_1
root@0d5f73b7ffc7:/# keactrl start
INFO/keactrl: Starting /usr/local/sbin/kea-dhcp4 -c /usr/local/etc/kea/kea-dhcp4.conf
INFO/keactrl: Starting /usr/local/sbin/kea-ctrl-agent -c /usr/local/etc/kea/kea-ctrl-agent.conf
root@0d5f73b7ffc7:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.3:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:50:37 GMT",
      "state": "waiting"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@0d5f73b7ffc7:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.2:8080/ | jq
curl: (7) Failed to connect to 172.18.0.2 port 8080: Connection refused
root@0d5f73b7ffc7:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.3:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:50:42 GMT",
      "state": "partner-down"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@0d5f73b7ffc7:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.2:8080/ | jq
curl: (7) Failed to connect to 172.18.0.2 port 8080: Connection refused

起動後，kea_1はstateが waitingから，partner-downにシフトしていることが見受けられる． kea_2に対しても同様のリクエストを行なっているが，kea_2ではまだkeaを起動していないためConnection refusedが返っている． kea_1を起動し，kea_2を起動していない場合，kea_1はこの状態に収束する．

ここでkea_2を起動する．

# kea_2
root@d864a28cbf84:/# keactrl start
INFO/keactrl: Starting /usr/local/sbin/kea-dhcp4 -c /usr/local/etc/kea/kea-dhcp4.conf
INFO/keactrl: Starting /usr/local/sbin/kea-ctrl-agent -c /usr/local/etc/kea/kea-ctrl-agent.conf
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.3:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:37 GMT",
      "state": "partner-down"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.2:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:41 GMT",
      "state": "waiting"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.3:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:47 GMT",
      "state": "partner-down"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.2:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:49 GMT",
      "state": "ready"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.3:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:54 GMT",
      "state": "hot-standby"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.2:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:55 GMT",
      "state": "ready"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.3:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:57 GMT",
      "state": "hot-standby"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]
root@d864a28cbf84:/# curl -sS -X POST -H "Content-Type: application/json" -d '{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }' http:///172.18.0.2:8080/ | jq
[
  {
    "arguments": {
      "date-time": "Fri, 25 Oct 2019 17:51:59 GMT",
      "state": "hot-standby"
    },
    "result": 0,
    "text": "HA peer status returned."
  }
]

ログを見てみる．

# kea_1
root@0d5f73b7ffc7:/# grep INFO /usr/local/var/log/kea-dhcp4.log | grep ha-hooks
2019-10-25 17:50:27.312 INFO  [kea-dhcp4.ha-hooks/1358] HA_CONFIGURATION_SUCCESSFUL HA hook library has been successfully configured
2019-10-25 17:50:27.312 INFO  [kea-dhcp4.ha-hooks/1358] HA_INIT_OK loading High Availability hooks library successful
2019-10-25 17:50:27.359 INFO  [kea-dhcp4.ha-hooks/1358] HA_LOCAL_DHCP_DISABLE local DHCP service is disabled while the kea_1 is in the WAITING state
2019-10-25 17:50:27.359 INFO  [kea-dhcp4.ha-hooks/1358] HA_SERVICE_STARTED started high availability service in hot-standby mode as primary server
2019-10-25 17:50:37.372 INFO  [kea-dhcp4.ha-hooks/1358] HA_STATE_TRANSITION server transitions from WAITING to PARTNER-DOWN state, partner state is UNDEFINED
2019-10-25 17:50:37.372 INFO  [kea-dhcp4.ha-hooks/1358] HA_LEASE_UPDATES_DISABLED lease updates will not be sent to the partner while in PARTNER-DOWN state
2019-10-25 17:50:37.372 INFO  [kea-dhcp4.ha-hooks/1358] HA_LOCAL_DHCP_ENABLE local DHCP service is enabled while the kea_1 is in the PARTNER-DOWN state
2019-10-25 17:51:52.821 INFO  [kea-dhcp4.ha-hooks/1358] HA_STATE_TRANSITION server transitions from PARTNER-DOWN to HOT-STANDBY state, partner state is READY
2019-10-25 17:51:52.821 INFO  [kea-dhcp4.ha-hooks/1358] HA_LEASE_UPDATES_ENABLED lease updates will be sent to the partner while in HOT-STANDBY state
2019-10-25 17:53:11.474 INFO  [kea-dhcp4.ha-hooks/1358] HA_DEINIT_OK unloading High Availability hooks library successful
root@0d5f73b7ffc7:/#

# kea_2
root@d864a28cbf84:/# grep INFO /usr/local/var/log/kea-dhcp4.log | grep ha-hooks
2019-10-25 17:51:33.880 INFO  [kea-dhcp4.ha-hooks/901] HA_CONFIGURATION_SUCCESSFUL HA hook library has been successfully configured
2019-10-25 17:51:33.880 INFO  [kea-dhcp4.ha-hooks/901] HA_INIT_OK loading High Availability hooks library successful
2019-10-25 17:51:33.928 INFO  [kea-dhcp4.ha-hooks/901] HA_LOCAL_DHCP_DISABLE local DHCP service is disabled while the kea_2 is in the WAITING state
2019-10-25 17:51:33.928 INFO  [kea-dhcp4.ha-hooks/901] HA_SERVICE_STARTED started high availability service in hot-standby mode as standby server
2019-10-25 17:51:45.738 INFO  [kea-dhcp4.ha-hooks/901] HA_STATE_TRANSITION server transitions from WAITING to SYNCING state, partner state is PARTNER-DOWN
2019-10-25 17:51:45.738 INFO  [kea-dhcp4.ha-hooks/901] HA_LEASE_UPDATES_DISABLED lease updates will not be sent to the partner while in SYNCING state
2019-10-25 17:51:45.738 INFO  [kea-dhcp4.ha-hooks/901] HA_SYNC_START starting lease database synchronization with kea_1
2019-10-25 17:51:45.743 INFO  [kea-dhcp4.ha-hooks/901] HA_LEASES_SYNC_LEASE_PAGE_RECEIVED received 0 leases from kea_1
2019-10-25 17:51:45.745 INFO  [kea-dhcp4.ha-hooks/901] HA_SYNC_SUCCESSFUL lease database synchronization with kea_1 completed successfully in 6.724 ms
2019-10-25 17:51:45.745 INFO  [kea-dhcp4.ha-hooks/901] HA_STATE_TRANSITION server transitions from SYNCING to READY state, partner state is PARTNER-DOWN
2019-10-25 17:51:45.745 INFO  [kea-dhcp4.ha-hooks/901] HA_LEASE_UPDATES_DISABLED lease updates will not be sent to the partner while in READY state
2019-10-25 17:51:56.910 INFO  [kea-dhcp4.ha-hooks/901] HA_STATE_TRANSITION server transitions from READY to HOT-STANDBY state, partner state is HOT-STANDBY
2019-10-25 17:51:56.911 INFO  [kea-dhcp4.ha-hooks/901] HA_LEASE_UPDATES_ENABLED lease updates will be sent to the partner while in HOT-STANDBY state
2019-10-25 17:51:56.911 INFO  [kea-dhcp4.ha-hooks/901] HA_LOCAL_DHCP_ENABLE local DHCP service is enabled while the kea_2 is in the HOT-STANDBY state
2019-10-25 17:53:12.373 INFO  [kea-dhcp4.ha-hooks/901] HA_DEINIT_OK unloading High Availability hooks library successful
root@d864a28cbf84:/#

これらをみてみると，起動直後，これらのHAは下記のような順序で状態遷移していることがわかる．

2019-10-25 17:50:27.359:: kea_1: waiting     , kea_2: UNDEFINED
2019-10-25 17:50:37.372:: kea_1: partner-down, kea_2: UNDEFINED
2019-10-25 17:51:33.928:: kea_1: partner-down, kea_2: waiting
2019-10-25 17:51:45.738:: kea_1: partner-down, kea_2: syncing
2019-10-25 17:51:45.745:: kea_1: partner-down, kea_2: ready
2019-10-25 17:51:52.821:: kea_1: hot-standby,  kea_2: ready
2019-10-25 17:51:56.910:: kea_1: hot-standby , kea_2: hot-standby

このそれぞれのstateの意味はkeaのdocumentを参照し，簡単に理解をまとめると下記の通りである．ほぼgoogle翻訳．

- Backup: backup-serverの平常状態で，この状態であればアクティブなサーバーからlease-updateを受信する．

- hot-standby: hot-standbyモードのアクティブサーバーの平常状態．hot-standbyモードにおけるプライマリサーバーとスタンバイサーバーは，通常この状態をとる．プライマリサーバーはDHCPクエリに応答し，スタンバイサーバーが存在する場合，スタンバイサーバーとバックアップサーバーにlease-updateを送信する

- load-balancing: load-balancingモードのアクティブなサーバーの平常状態．loadbalancingモードにおけるプライマリサーバーとセカンダリサーバーは，通常のこの状態をとる．両方のサーバーがDHCPクエリに応答し，リース更新を相互に送信し，バックアップサーバーが存在する場合にはバックアップサーバーに送信する．

- partner-down: partnerがオフラインであることを検出すると，アクティブサーバーはこの状態に移行する．バックアップサーバが利用できなくともサーバーはこの状態に移行しない．パートナーダウン状態では，サーバーはすべてのDHCPクエリに応答し，'通常時に現在使用できないpartnerによって処理されるべきクエリ'にも応答する．

- ready: lease-databaseをアクティブなパートナーと同期した後、アクティブなサーバーはこの状態となる．この状態は、パートナーに通知される可能性がある（partner-down状態にある可能性が高いため、通常の操作に戻る可能性が高い．その場合，ready 状態にあるサーバーも続けて通常の動作を開始する．)

- syncing: アクティブなサーバーはこの状態に移行して，アクティブなパートナーからリースを取得し，ローカルリースデータベースを更新します．この状態になると，dhcp-disableを発行して，リースの取得元のパートナーのDHCPサービスを無効にします． DHCPサービスは最大60秒間無効になります．その後，同期パートナーが再び停止し，サービスを再度有効にできない場合に備えて，自動的に有効になります．同期が完了すると，同期サーバーはdhcp-enableを発行して，パートナーのDHCPサービスを再度有効にします．同期操作は同期です．サーバーは，パートナーからの応答を待っており，リースの同期が行われている間は何もしていません．データベースをパートナーと同期しないように構成されているサーバー，つまりsync-leases構成パラメーターがfalseに設定されている場合，この状態に移行することはありません．代わりに，待機状態から準備完了状態に直接移行します．

- terminated: ha-hooksライブラリがHA構成を組めず，問題を解決するために手動オペレーションが必要な場合にアクティブサーバーはこの状態となる．HAセットアップの様々なの問題により，サーバーがこの状態になる可能性がある．． Kea 1.4.0リリースの時点でのHAを終了させる唯一の問題は，各サーバーのクロックが60秒以上離れている場合である．terminated 状態のサーバーは選択されたHAモード（ロードバランシングまたはホットスタンバイ）に基づいてDHCPクライアントに応答し続けますが，lease-updateは交換されず，ha-heartbeatは送信されない．terminated状態になると，サーバーは再起動されるまでこの状態となる．オペレータはサーバーを再起動する前に，根本原因を解決する必要がある．

- wainting: 起動された各サーバーはこの状態となる．バックアップサーバーは，この状態からバックアップ状態に直接移行する．アクティブなサーバーは，その状態を確認するためにpartnerにha-heartbeatを送信する．pertnerがオフラインであると考えられる場合，サーバーはpartner-down状態に移行する．そうでない場合，sync-leases構成パラメーターの設定に応じてsyncing状態またはready状態に移行する．両方のサーバーがwaiting状態（同時起動）にあると思われる場合，プライマリサーバーが先に状態遷移する．プライマリサーバーがready状態に移行するまで，セカンダリサーバーまたはスタンバイサーバーはwaiting状態のままとなる．

hot-stanbuyモードでのDHCPリース動作

DHCPリースを模擬するために，今回はdhtestを利用した．

構築してhot-standby HA構成のサーバ群が接続されているNW上に存在するdhtest用VMで，下記の通りコマンドを実行する．

./dhtest -i <interface> -m <mac_addr>

mac_addr はdhtestホストから送出するDHCPパケットのchaddr フィールドに入るMACアドレスと思えばよい．これを数回繰り返し実施した時の振る舞いを見てみる． dhtestホストでは下記のようにアドレスリースが模擬できているようである．

[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:33
DHCP discover sent   - Client MAC : 00:00:00:11:22:33
DHCP offer received  - Offered IP : 172.18.1.1
DHCP request sent  - Client MAC : 00:00:00:11:22:33
^[[ADHCP ack received  - Acquired IP: 172.18.1.1
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:34
DHCP discover sent   - Client MAC : 00:00:00:11:22:34
DHCP offer received  - Offered IP : 172.18.1.2
DHCP request sent  - Client MAC : 00:00:00:11:22:34
DHCP ack received  - Acquired IP: 172.18.1.2
[root@a50202e240fb dhtest-master]# ./dhtest -i eth0 -m 00:00:00:11:22:35
DHCP discover sent   - Client MAC : 00:00:00:11:22:35
DHCP offer received  - Offered IP : 172.18.1.3
DHCP request sent  - Client MAC : 00:00:00:11:22:35
DHCP ack received  - Acquired IP: 172.18.1.3
[root@a50202e240fb dhtest-master]#

この時，kea_1 とkea_2 ホストでパケットキャプチャを行うと．

## kea_1
root@0d5f73b7ffc7:~# tcpdump -i eth0 port 67 or  port 68 -v -n
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^[[O06:44:35.308246 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:33, length 251, xid 0x6a521a43, Flags [none]
          Client-Ethernet-Address 00:00:00:11:22:33
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Parameter-Request Option 55, length 5:
              Subnet-Mask, BR, Default-Gateway, Domain-Name
              Domain-Name-Server
06:44:35.311244 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    172.18.0.3.67 > 172.18.1.1.68: BOOTP/DHCP, Reply, length 290, xid 0x6a521a43, Flags [none]
          Your-IP 172.18.1.1
          Client-Ethernet-Address 00:00:00:11:22:33
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            Subnet-Mask Option 1, length 4: 255.255.0.0
            Default-Gateway Option 3, length 4: 172.18.0.1
            Domain-Name-Server Option 6, length 8: 8.8.8.8,8.8.4.4
            Lease-Time Option 51, length 4: 3600
            DHCP-Message Option 53, length 1: Offer
            Server-ID Option 54, length 4: 172.18.0.3
            RN Option 58, length 4: 1200
            RB Option 59, length 4: 2400
06:44:35.311902 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:33, length 263, xid 0x6a521a43, Flags [none]
          Client-Ethernet-Address 00:00:00:11:22:33
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Request
            Requested-IP Option 50, length 4: 172.18.1.1
            Server-ID Option 54, length 4: 172.18.0.3
            Parameter-Request Option 55, length 5:
              Subnet-Mask, BR, Default-Gateway, Domain-Name
              Domain-Name-Server
06:44:36.316180 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    172.18.0.3.67 > 172.18.1.1.68: BOOTP/DHCP, Reply, length 290, xid 0x6a521a43, Flags [none]
          Your-IP 172.18.1.1
          Client-Ethernet-Address 00:00:00:11:22:33
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            Subnet-Mask Option 1, length 4: 255.255.0.0
            Default-Gateway Option 3, length 4: 172.18.0.1
            Domain-Name-Server Option 6, length 8: 8.8.8.8,8.8.4.4
            Lease-Time Option 51, length 4: 3600
            DHCP-Message Option 53, length 1: ACK
            Server-ID Option 54, length 4: 172.18.0.3
            RN Option 58, length 4: 1200
            RB Option 59, length 4: 2400
06:44:38.007073 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:34, length 251, xid 0x23552c97, Flags [none]
          Client-Ethernet-Address 00:00:00:11:22:34
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Parameter-Request Option 55, length 5:
              Subnet-Mask, BR, Default-Gateway, Domain-Name
              Domain-Name-Server
06:44:38.009206 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    172.18.0.3.67 > 172.18.1.2.68: BOOTP/DHCP, Reply, length 290, xid 0x23552c97, Flags [none]
          Your-IP 172.18.1.2
          Client-Ethernet-Address 00:00:00:11:22:34
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            Subnet-Mask Option 1, length 4: 255.255.0.0
            Default-Gateway Option 3, length 4: 172.18.0.1
            Domain-Name-Server Option 6, length 8: 8.8.8.8,8.8.4.4
            Lease-Time Option 51, length 4: 3600
            DHCP-Message Option 53, length 1: Offer
            Server-ID Option 54, length 4: 172.18.0.3
            RN Option 58, length 4: 1200
            RB Option 59, length 4: 2400
06:44:38.009418 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:34, length 263, xid 0x23552c97, Flags [none]
          Client-Ethernet-Address 00:00:00:11:22:34
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Request
            Requested-IP Option 50, length 4: 172.18.1.2
            Server-ID Option 54, length 4: 172.18.0.3
            Parameter-Request Option 55, length 5:
              Subnet-Mask, BR, Default-Gateway, Domain-Name
              Domain-Name-Server
06:44:39.014721 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    172.18.0.3.67 > 172.18.1.2.68: BOOTP/DHCP, Reply, length 290, xid 0x23552c97, Flags [none]
          Your-IP 172.18.1.2
          Client-Ethernet-Address 00:00:00:11:22:34
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            Subnet-Mask Option 1, length 4: 255.255.0.0
            Default-Gateway Option 3, length 4: 172.18.0.1
            Domain-Name-Server Option 6, length 8: 8.8.8.8,8.8.4.4
            Lease-Time Option 51, length 4: 3600
            DHCP-Message Option 53, length 1: ACK
            Server-ID Option 54, length 4: 172.18.0.3
            RN Option 58, length 4: 1200
            RB Option 59, length 4: 2400
06:44:40.623865 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:35, length 251, xid 0x4a57a2d2, Flags [none]
          Client-Ethernet-Address 00:00:00:11:22:35
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Parameter-Request Option 55, length 5:
              Subnet-Mask, BR, Default-Gateway, Domain-Name
              Domain-Name-Server
06:44:40.625059 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    172.18.0.3.67 > 172.18.1.3.68: BOOTP/DHCP, Reply, length 290, xid 0x4a57a2d2, Flags [none]
          Your-IP 172.18.1.3
          Client-Ethernet-Address 00:00:00:11:22:35
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            Subnet-Mask Option 1, length 4: 255.255.0.0
            Default-Gateway Option 3, length 4: 172.18.0.1
            Domain-Name-Server Option 6, length 8: 8.8.8.8,8.8.4.4
            Lease-Time Option 51, length 4: 3600
            DHCP-Message Option 53, length 1: Offer
            Server-ID Option 54, length 4: 172.18.0.3
            RN Option 58, length 4: 1200
            RB Option 59, length 4: 2400
06:44:40.625322 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:35, length 263, xid 0x4a57a2d2, Flags [none]
          Client-Ethernet-Address 00:00:00:11:22:35
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Request
            Requested-IP Option 50, length 4: 172.18.1.3
            Server-ID Option 54, length 4: 172.18.0.3
            Parameter-Request Option 55, length 5:
              Subnet-Mask, BR, Default-Gateway, Domain-Name
              Domain-Name-Server
06:44:42.630578 IP (tos 0x10, ttl 128, id 0, offset 0, flags [DF], proto UDP (17), length 318)
    172.18.0.3.67 > 172.18.1.3.68: BOOTP/DHCP, Reply, length 290, xid 0x4a57a2d2, Flags [none]
          Your-IP 172.18.1.3
          Client-Ethernet-Address 00:00:00:11:22:35
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            Subnet-Mask Option 1, length 4: 255.255.0.0
            Default-Gateway Option 3, length 4: 172.18.0.1
            Domain-Name-Server Option 6, length 8: 8.8.8.8,8.8.4.4
            Lease-Time Option 51, length 4: 3600
            DHCP-Message Option 53, length 1: ACK
            Server-ID Option 54, length 4: 172.18.0.3
            RN Option 58, length 4: 1200
            RB Option 59, length 4: 2400

## kea_2
root@d864a28cbf84:~# tcpdump -i eth0 port 67 or port 68 -v -n
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:44:35.308297 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:33, length 251, xid 0x6a521a43, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:33
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:44:35.311922 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:33, length 263, xid 0x6a521a43, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:33
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.1.1
      Server-ID Option 54, length 4: 172.18.0.3
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:44:38.007102 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:34, length 251, xid 0x23552c97, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:34
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:44:38.009439 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:34, length 263, xid 0x23552c97, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:34
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.1.2
      Server-ID Option 54, length 4: 172.18.0.3
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:44:40.623903 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 279)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:35, length 251, xid 0x4a57a2d2, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:35
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Discover
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server
06:44:40.625346 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 291)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:11:22:35, length 263, xid 0x4a57a2d2, Flags [none]
    Client-Ethernet-Address 00:00:00:11:22:35
    Vendor-rfc1048 Extensions
      Magic Cookie 0x63825363
      DHCP-Message Option 53, length 1: Request
      Requested-IP Option 50, length 4: 172.18.1.3
      Server-ID Option 54, length 4: 172.18.0.3
      Parameter-Request Option 55, length 5:
        Subnet-Mask, BR, Default-Gateway, Domain-Name
        Domain-Name-Server

これら2つのパケットキャプチャをみて理解できるとおり， kea_1はクライアントからの3つ全てのDHCP要求(DISCOVERY, REQUEST)において，クライアントに対してOFFER, ACK を返しており，kea_2はクライアントからの要求には全く応答パケットを送出していないことがわかる．これはHAモードのうち，primaryの1台のみがDHCPに応答するhot-standbyの動作となっていることがわかる．

またこの時のlease-updateを見てみよう．

## kea_1

root@0d5f73b7ffc7:/# tcpdump -i eth0 port 8080 -v -n
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:44:35.313735 IP (tos 0x0, ttl 64, id 57517, offset 0, flags [DF], proto TCP (6), length 403)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x59af (incorrect -> 0x066f), seq 220565762:220566113, ack 2766104306, win 254, options [nop,nop,TS val 26311210 ecr 26310630], length 351: HTTP, length: 351
  POST / HTTP/1.1
  Content-Length: 279
  Content-Type: application/json

  { "arguments": { "expire": 1572335075, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:33", "ip-address": "172.18.1.1", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:44:35.315597 IP (tos 0x0, ttl 64, id 2855, offset 0, flags [DF], proto TCP (6), length 208)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x58ec (incorrect -> 0x0bf1), seq 1:157, ack 351, win 235, options [nop,nop,TS val 26311210 ecr 26311210], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:35 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:44:35.315738 IP (tos 0x0, ttl 64, id 57518, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xd5cc), ack 157, win 262, options [nop,nop,TS val 26311211 ecr 26311210], length 0
06:44:38.012408 IP (tos 0x0, ttl 64, id 57519, offset 0, flags [DF], proto TCP (6), length 403)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x59af (incorrect -> 0xfd19), seq 351:702, ack 157, win 262, options [nop,nop,TS val 26311480 ecr 26311210], length 351: HTTP, length: 351
  POST / HTTP/1.1
  Content-Length: 279
  Content-Type: application/json

  { "arguments": { "expire": 1572335077, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:34", "ip-address": "172.18.1.2", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:44:38.015389 IP (tos 0x0, ttl 64, id 2856, offset 0, flags [DF], proto TCP (6), length 208)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x58ec (incorrect -> 0x07cf), seq 157:313, ack 702, win 243, options [nop,nop,TS val 26311480 ecr 26311480], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:38 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:44:38.015425 IP (tos 0x0, ttl 64, id 57520, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xd1ae), ack 313, win 270, options [nop,nop,TS val 26311480 ecr 26311480], length 0
06:44:40.625311 IP (tos 0x0, ttl 64, id 57521, offset 0, flags [DF], proto TCP (6), length 176)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x58cc (incorrect -> 0x473e), seq 702:826, ack 313, win 270, options [nop,nop,TS val 26311745 ecr 26311480], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:44:40.632575 IP (tos 0x0, ttl 64, id 2857, offset 0, flags [DF], proto TCP (6), length 303)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x594b (incorrect -> 0x866a), seq 313:564, ack 826, win 243, options [nop,nop,TS val 26311746 ecr 26311745], length 251: HTTP, length: 251
  HTTP/1.1 200 OK
  Content-Length: 142
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:40 GMT

  [ { "arguments": { "date-time": "Tue, 29 Oct 2019 06:44:40 GMT", "state": "hot-standby" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:44:40.632606 IP (tos 0x0, ttl 64, id 57522, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xce1a), ack 564, win 279, options [nop,nop,TS val 26311746 ecr 26311746], length 0
06:44:41.628464 IP (tos 0x0, ttl 64, id 57523, offset 0, flags [DF], proto TCP (6), length 403)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x59af (incorrect -> 0xfb10), seq 826:1177, ack 564, win 279, options [nop,nop,TS val 26311845 ecr 26311746], length 351: HTTP, length: 351
  POST / HTTP/1.1
  Content-Length: 279
  Content-Type: application/json

  { "arguments": { "expire": 1572335080, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:35", "ip-address": "172.18.1.3", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:44:41.633534 IP (tos 0x0, ttl 64, id 2858, offset 0, flags [DF], proto TCP (6), length 208)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x58ec (incorrect -> 0x0080), seq 564:720, ack 1177, win 252, options [nop,nop,TS val 26311846 ecr 26311845], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:41 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:44:41.633621 IP (tos 0x0, ttl 64, id 57524, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xcb4f), ack 720, win 287, options [nop,nop,TS val 26311846 ecr 26311846], length 0
06:44:42.637356 IP (tos 0x0, ttl 64, id 43254, offset 0, flags [DF], proto TCP (6), length 176)
    172.18.0.2.42824 > 172.18.0.3.8080: Flags [P.], cksum 0x58cc (incorrect -> 0x7e2b), seq 1837942929:1837943053, ack 2537261301, win 254, options [nop,nop,TS val 26311946 ecr 26310830], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:44:42.640120 IP (tos 0x0, ttl 64, id 59900, offset 0, flags [DF], proto TCP (6), length 303)
    172.18.0.3.8080 > 172.18.0.2.42824: Flags [P.], cksum 0x594b (incorrect -> 0xb605), seq 1:252, ack 124, win 227, options [nop,nop,TS val 26311946 ecr 26311946], length 251: HTTP, length: 251
  HTTP/1.1 200 OK
  Content-Length: 142
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:42 GMT

  [ { "arguments": { "date-time": "Tue, 29 Oct 2019 06:44:42 GMT", "state": "hot-standby" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:44:42.640722 IP (tos 0x0, ttl 64, id 43255, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.2.42824 > 172.18.0.3.8080: Flags [.], cksum 0x5850 (incorrect -> 0x01b8), ack 252, win 262, options [nop,nop,TS val 26311946 ecr 26311946], length 0

## kea_2

root@d864a28cbf84:/# tcpdump -i eth0 port 8080 -v -n
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:44:35.313767 IP (tos 0x0, ttl 64, id 57517, offset 0, flags [DF], proto TCP (6), length 403)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x59af (incorrect -> 0x066f), seq 220565762:220566113, ack 2766104306, win 254, options [nop,nop,TS val 26311210 ecr 26310630], length 351: HTTP, length: 351
  POST / HTTP/1.1
  Content-Length: 279
  Content-Type: application/json

  { "arguments": { "expire": 1572335075, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:33", "ip-address": "172.18.1.1", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:44:35.315569 IP (tos 0x0, ttl 64, id 2855, offset 0, flags [DF], proto TCP (6), length 208)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x58ec (incorrect -> 0x0bf1), seq 1:157, ack 351, win 235, options [nop,nop,TS val 26311210 ecr 26311210], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:35 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:44:35.315755 IP (tos 0x0, ttl 64, id 57518, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xd5cc), ack 157, win 262, options [nop,nop,TS val 26311211 ecr 26311210], length 0
06:44:38.012495 IP (tos 0x0, ttl 64, id 57519, offset 0, flags [DF], proto TCP (6), length 403)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x59af (incorrect -> 0xfd19), seq 351:702, ack 157, win 262, options [nop,nop,TS val 26311480 ecr 26311210], length 351: HTTP, length: 351
  POST / HTTP/1.1
  Content-Length: 279
  Content-Type: application/json

  { "arguments": { "expire": 1572335077, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:34", "ip-address": "172.18.1.2", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:44:38.015323 IP (tos 0x0, ttl 64, id 2856, offset 0, flags [DF], proto TCP (6), length 208)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x58ec (incorrect -> 0x07cf), seq 157:313, ack 702, win 243, options [nop,nop,TS val 26311480 ecr 26311480], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:38 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:44:38.015447 IP (tos 0x0, ttl 64, id 57520, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xd1ae), ack 313, win 270, options [nop,nop,TS val 26311480 ecr 26311480], length 0
06:44:40.625346 IP (tos 0x0, ttl 64, id 57521, offset 0, flags [DF], proto TCP (6), length 176)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x58cc (incorrect -> 0x473e), seq 702:826, ack 313, win 270, options [nop,nop,TS val 26311745 ecr 26311480], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:44:40.632539 IP (tos 0x0, ttl 64, id 2857, offset 0, flags [DF], proto TCP (6), length 303)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x594b (incorrect -> 0x866a), seq 313:564, ack 826, win 243, options [nop,nop,TS val 26311746 ecr 26311745], length 251: HTTP, length: 251
  HTTP/1.1 200 OK
  Content-Length: 142
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:40 GMT

  [ { "arguments": { "date-time": "Tue, 29 Oct 2019 06:44:40 GMT", "state": "hot-standby" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:44:40.632623 IP (tos 0x0, ttl 64, id 57522, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xce1a), ack 564, win 279, options [nop,nop,TS val 26311746 ecr 26311746], length 0
06:44:41.628608 IP (tos 0x0, ttl 64, id 57523, offset 0, flags [DF], proto TCP (6), length 403)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [P.], cksum 0x59af (incorrect -> 0xfb10), seq 826:1177, ack 564, win 279, options [nop,nop,TS val 26311845 ecr 26311746], length 351: HTTP, length: 351
  POST / HTTP/1.1
  Content-Length: 279
  Content-Type: application/json

  { "arguments": { "expire": 1572335080, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:35", "ip-address": "172.18.1.3", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }[!http]
06:44:41.633418 IP (tos 0x0, ttl 64, id 2858, offset 0, flags [DF], proto TCP (6), length 208)
    172.18.0.2.8080 > 172.18.0.3.33988: Flags [P.], cksum 0x58ec (incorrect -> 0x0080), seq 564:720, ack 1177, win 252, options [nop,nop,TS val 26311846 ecr 26311845], length 156: HTTP, length: 156
  HTTP/1.1 200 OK
  Content-Length: 48
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:41 GMT

  [ { "result": 0, "text": "IPv4 lease added." } ][!http]
06:44:41.633649 IP (tos 0x0, ttl 64, id 57524, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.3.33988 > 172.18.0.2.8080: Flags [.], cksum 0x5850 (incorrect -> 0xcb4f), ack 720, win 287, options [nop,nop,TS val 26311846 ecr 26311846], length 0
06:44:42.637308 IP (tos 0x0, ttl 64, id 43254, offset 0, flags [DF], proto TCP (6), length 176)
    172.18.0.2.42824 > 172.18.0.3.8080: Flags [P.], cksum 0x58cc (incorrect -> 0x7e2b), seq 1837942929:1837943053, ack 2537261301, win 254, options [nop,nop,TS val 26311946 ecr 26310830], length 124: HTTP, length: 124
  POST / HTTP/1.1
  Content-Length: 53
  Content-Type: application/json

  { "command": "ha-heartbeat", "service": [ "dhcp4" ] }[!http]
06:44:42.640413 IP (tos 0x0, ttl 64, id 59900, offset 0, flags [DF], proto TCP (6), length 303)
    172.18.0.3.8080 > 172.18.0.2.42824: Flags [P.], cksum 0x594b (incorrect -> 0xb605), seq 1:252, ack 124, win 227, options [nop,nop,TS val 26311946 ecr 26311946], length 251: HTTP, length: 251
  HTTP/1.1 200 OK
  Content-Length: 142
  Content-Type: application/json
  Date: Tue, 29 Oct 2019 06:44:42 GMT

  [ { "arguments": { "date-time": "Tue, 29 Oct 2019 06:44:42 GMT", "state": "hot-standby" }, "result": 0, "text": "HA peer status returned." } ][!http]
06:44:42.640696 IP (tos 0x0, ttl 64, id 43255, offset 0, flags [DF], proto TCP (6), length 52)
    172.18.0.2.42824 > 172.18.0.3.8080: Flags [.], cksum 0x5850 (incorrect -> 0x01b8), ack 252, win 262, options [nop,nop,TS val 26311946 ecr 26311946], length 0

上記より，kea_1からkea_2に向けて

{ "arguments": { "expire": 1572335075, "force-create": true, "fqdn-fwd": false, "fqdn-rev": false, "hostname": "", "hw-address": "00:00:00:11:22:33", "ip-address": "172.18.1.1", "state": 0, "subnet-id": 1, "valid-lft": 3600 }, "command": "lease4-update", "service": [ "dhcp4" ] }

というjsonフォーマットでlease-updateが送出され，

{ "result": 0, "text": "IPv4 lease added." }

という形で応答していることがわかる． Kea DHCPにおいてはこのようにREST APIを用いてlease-updateの交換を行うことが特徴の一つであると言えるだろう．

まとめ

というわけで，無事Kea DHCPを使ってhot-standbyモードのHA構成を組むことができました．本記事では以下のことを実施しました．

ISC DHCPの後継であるKea DHCPについて紹介し，冗長構成の違いを紹介した．
ISCのKea DHCPを用いてhot-standbyモードのHA構成を組むことについて紹介した．
hot-standyモードのHAにおいて実際にクライアントを模擬してDHCPによるアドレスリースを行い，2つのサーバの応答の違いをパケットキャプチャによって確認した．
HA構成における2サーバ間のlease-updateについて，REST APIによって行われていることを確認した．

また，次回以降では，load-balancingモードでのHA構成の構築と，hot-standby構成との振る舞いの違いや，HA構成を取ることによるパフォーマンス影響，HA構成で本当に冗長化ができているのか，等について見てみようと考えています．

References

P.S.

この内容をQiitaに投稿してみました: Kea DHCPでAct-Sby(hot-standby)のHA構成を組む - Qiita

2019-10-24

Cisco Aironet APの動作モードを変更する

network

概要

Cisco AironetシリーズのAPにおけるAPモードにはWLCを利用してconfigurationを行い動作する'Lightweight Mode'と，単体でスタンドアロンモードのように自律動作する'Autonomous Mode'が存在する．

この記事ではCisco Aironetシリーズのモード変更を行う際の手順やメモを紹介する．基本的には下記を参照している．
Lightweight - Autonomous AP変換方法 - Cisco Community

はじめに

Cisco AironetシリーズのAPは，Lightweight ModeとAutonomous Modeの2つのモードをもち，Lightweight ModeではWCLとトンネルを張り，WLCからconfigをLoadし，その後もWLCからのコマンドにより動作をするが，Autonomous Modeでは通常のSW等と同様にAP内部にconfigをもち，その設定をもとに動作する．今回はCisco AironetシリーズのAIR-CAP1702I-Q-K9に対して操作を実施した．

Lightweight ModeとAutonomous Modeの変更方法

購入直後は，Lightweight Mode, Autonomous Mode両方のconfigがAPに内包されているようである．出荷状態であれば，何もしなければLightweight Modeで動作し，これをAutonomous Modeで動かしたい場合は下記のコマンドによりコンバートを行うことができる.
AP 1700 OS コンバートの仕方 - Cisco Community

ap# capwap ap autonomous

なお，このコマンドを入力した場合，2つあったLightweight ModeのイメージとAutonomous Modeのイメージのうち，Lightweightモードのイメージは削除されてしまうめ注意が必要だ．このコマンドでAutonomous ModeにしたAPを，Lightweight Modeに戻したい場合等は下記に示す手順によってFirmwareを入れ替える必要がある．

Lightweight ModeとAutonomous Modeのイメージの違い

Lightweight - Autonomous AP変換方法 - Cisco Communityによると，IOSの記号番号によりIOSの種別が判別できるそうだ．

RCVK9W8      リカバリ IOS (工場出荷時にインストールされている集中管理型用 IOS)
K9W8         集中管理型 AP (Lightweight AP) IOS
K9W7         自律型 AP (Autonomous AP) IOS

たとえば下記のようなOSのAPがあるとする．

ap# show version | inc IOS
Cisco IOS Software, C1700 Software (AP3G2-K9W7-M), Version 15.3(3)JH, RELEASE SOFTWARE (fc3)

するとこれはK9W7に該当するのでAutonomous Modeで動作していることがわかる．

Lightweight ModeからAutonomous Modeに切り替える場合はAutonomous Modeを示すK9W7が記載されているFirmwareを，Autonomous ModeからLightweight Modeに切り替える場合はLightweight Modeを示すK9W8が記載されたFirmwareをそれぞれ事前に入手しておくことが必要である．

Lightweight ModeからAutonomous Modeへの変更

基本的にはコンソールでのログイン後，下記のコマンド群で実施が可能である．

ap# capwap ap ip address <ip_addr> <mask>
ap# capwap ap ip default-gateway <gateway_ip>
ap# show capwap ip config

ap# debug capwap console cli
ap# debug capwap client no-reload
ap# archive download-sw /create-space /overwrite tftp:<path_to_file> // K9W7(Autonomous Mode)のFirmを指定する．
ap# dir flash:
ap# show flash
ap# show boot
ap# verify <path_to_file>
ap# reload

上から追っていく．まず，APへのIP address, gateway等の設定を行う．これはFirmwareを流し込む上でftp, tftp等を利用する場合にL3疎通性を確保するためである．その後debugモードを有効にし，archiveコマンドが打てる状態になる．このdebugモードを有効にしていないとarchiveコマンドは正常に発行できないので注意する．その後archiveコマンドを用いてftp/tftp等でFirmwareを転送する．create-space，overwrite等のoptionsは状況/好みに応じて付与するとよいだろう．転送が終了したら，show flash, show boot等のコマンドを用いてboot imageが正常に転送されているか，所望の通り変更されているかを確認する．最後にreloadを実施すると，次回そのimageでbootする．転送後に必要に応じてverifyコマンド等でmd5をチェックしておくと安心である．

Autonomous Mode からLightweight Modeへの変更

上記とほぼ同様である．流れとしてはL3疎通性を確保し，archiveコマンドで書き込み，確認後reloadを行う．

ap(config)# interface bvi 1
ap(config-if)# ip address <ip_addr> <mask>
ap(config-if)# no shutdown
ap(config-if)# interface GigabitEthernet1
ap(config-if)# no shutdown
ap(config-if)# exit
ap(config)# ip default-gateway <gateway_ip>
ap(config)# end

ap# archive download-sw /create-space /overwrite tftp:<path_to_file> // K9W8(Lightweight Mode)のFirmを指定する．
ap# dir flash:
ap# show flash
ap# show boot
ap# reload

前半のip指定については，Autonomous Modeで運用している場合は多くの場合すでに付与されていることがほとんどであると考えられるため，多くの状況で省略可能であると思われる．後の手順はLightweight ModeからAutonomous Modeに変更する手順とほぼ変わらない．

まとめ

Cisco AironetシリーズのAPにおけるAPモードにはWLCを利用してconfigurationを行い動作する'Lightweight Mode'と，単体でスタンドアロンモードのように自律動作する'Autonomous Mode'が存在することを紹介した．
Lightweight Mode, Autonomous Mode双方のモード切り替えをFirmware Imageを入れ替えることで実施する例を紹介した．

おまけ - MacOSでtftpサーバを動かす．

# 起動
sudo launchctl load -w /System/Library/LaunchDaemons/tftp.plist
# 終了
sudo launchctl unload -w /System/Library/LaunchDaemons/tftp.plist
# portが空いているか確認
sudo lsof -i:69

## MEMO
# /private/tftpboot 配下が転送ディレクトリとなる．permissionに注意する．
# イメージをputする場合には/private/tftpboot 配下に同名の空ファイルが必要なようである(createの権限不足？)．
# workaroundとしては，touch コマンド等を利用してからファイルを作成．chmodを実施し適切にpermissionを設定するとよい．