Ubuntu server 18.04 (bionic) network settings broken after power cycle












1















Networking was fine before reboot. After reboot, the system and all network settings are incorrect.



Notes:




  • Ubuntu Server 18.04 (Bionic)

  • Canonical livepatch was set up a few days ago.

  • The new network thingy, netplan, doesn't seem to be installed (however there is a file /etc/netplan/50-cloud-init.yaml)

  • The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).

  • The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:


  • /etc/resolv.conf is symlinked to /run/systemd/resolve/stub-resolv.conf file. When I update it manually with nameserver 192.168.1.1 it works.


ping 192.168.1.1
connect: Network is unreachable

ping google.com
ping: google.com: System error

dig google.com
connection timed out ...
...

ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
ipconfig enp7s0 up
ping 192.168.1.1
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...

nslookup google.com
Server: 192.168.1.1
Address: 192.168.1.1#53

Non-authoritative answer:
Name google.com
Address 172.217.164.110
...

ping google.com
ping: google.com: System error # ... still network issue, most tools do not resolve hosts


systemd-resolve google.com
google.com: 216.58.195.238

-- Information acquired via protocol DNS in 3.0494s.
-- Data is authenticated: no




I've also tried configuring /etc/resolv.conf /etc/nsswitch.conf to no avail.



And on reboot, it seems that the network managers, maybe systemd-networkd and systemd-resolved reset some of the /etc files to settings that don't work.



Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.










share|improve this question





























    1















    Networking was fine before reboot. After reboot, the system and all network settings are incorrect.



    Notes:




    • Ubuntu Server 18.04 (Bionic)

    • Canonical livepatch was set up a few days ago.

    • The new network thingy, netplan, doesn't seem to be installed (however there is a file /etc/netplan/50-cloud-init.yaml)

    • The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).

    • The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:


    • /etc/resolv.conf is symlinked to /run/systemd/resolve/stub-resolv.conf file. When I update it manually with nameserver 192.168.1.1 it works.


    ping 192.168.1.1
    connect: Network is unreachable

    ping google.com
    ping: google.com: System error

    dig google.com
    connection timed out ...
    ...

    ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
    ipconfig enp7s0 up
    ping 192.168.1.1
    64 bytes from 192.168.1.1: ...
    64 bytes from 192.168.1.1: ...
    64 bytes from 192.168.1.1: ...

    nslookup google.com
    Server: 192.168.1.1
    Address: 192.168.1.1#53

    Non-authoritative answer:
    Name google.com
    Address 172.217.164.110
    ...

    ping google.com
    ping: google.com: System error # ... still network issue, most tools do not resolve hosts


    systemd-resolve google.com
    google.com: 216.58.195.238

    -- Information acquired via protocol DNS in 3.0494s.
    -- Data is authenticated: no




    I've also tried configuring /etc/resolv.conf /etc/nsswitch.conf to no avail.



    And on reboot, it seems that the network managers, maybe systemd-networkd and systemd-resolved reset some of the /etc files to settings that don't work.



    Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.










    share|improve this question



























      1












      1








      1








      Networking was fine before reboot. After reboot, the system and all network settings are incorrect.



      Notes:




      • Ubuntu Server 18.04 (Bionic)

      • Canonical livepatch was set up a few days ago.

      • The new network thingy, netplan, doesn't seem to be installed (however there is a file /etc/netplan/50-cloud-init.yaml)

      • The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).

      • The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:


      • /etc/resolv.conf is symlinked to /run/systemd/resolve/stub-resolv.conf file. When I update it manually with nameserver 192.168.1.1 it works.


      ping 192.168.1.1
      connect: Network is unreachable

      ping google.com
      ping: google.com: System error

      dig google.com
      connection timed out ...
      ...

      ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
      ipconfig enp7s0 up
      ping 192.168.1.1
      64 bytes from 192.168.1.1: ...
      64 bytes from 192.168.1.1: ...
      64 bytes from 192.168.1.1: ...

      nslookup google.com
      Server: 192.168.1.1
      Address: 192.168.1.1#53

      Non-authoritative answer:
      Name google.com
      Address 172.217.164.110
      ...

      ping google.com
      ping: google.com: System error # ... still network issue, most tools do not resolve hosts


      systemd-resolve google.com
      google.com: 216.58.195.238

      -- Information acquired via protocol DNS in 3.0494s.
      -- Data is authenticated: no




      I've also tried configuring /etc/resolv.conf /etc/nsswitch.conf to no avail.



      And on reboot, it seems that the network managers, maybe systemd-networkd and systemd-resolved reset some of the /etc files to settings that don't work.



      Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.










      share|improve this question
















      Networking was fine before reboot. After reboot, the system and all network settings are incorrect.



      Notes:




      • Ubuntu Server 18.04 (Bionic)

      • Canonical livepatch was set up a few days ago.

      • The new network thingy, netplan, doesn't seem to be installed (however there is a file /etc/netplan/50-cloud-init.yaml)

      • The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).

      • The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:


      • /etc/resolv.conf is symlinked to /run/systemd/resolve/stub-resolv.conf file. When I update it manually with nameserver 192.168.1.1 it works.


      ping 192.168.1.1
      connect: Network is unreachable

      ping google.com
      ping: google.com: System error

      dig google.com
      connection timed out ...
      ...

      ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
      ipconfig enp7s0 up
      ping 192.168.1.1
      64 bytes from 192.168.1.1: ...
      64 bytes from 192.168.1.1: ...
      64 bytes from 192.168.1.1: ...

      nslookup google.com
      Server: 192.168.1.1
      Address: 192.168.1.1#53

      Non-authoritative answer:
      Name google.com
      Address 172.217.164.110
      ...

      ping google.com
      ping: google.com: System error # ... still network issue, most tools do not resolve hosts


      systemd-resolve google.com
      google.com: 216.58.195.238

      -- Information acquired via protocol DNS in 3.0494s.
      -- Data is authenticated: no




      I've also tried configuring /etc/resolv.conf /etc/nsswitch.conf to no avail.



      And on reboot, it seems that the network managers, maybe systemd-networkd and systemd-resolved reset some of the /etc files to settings that don't work.



      Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.







      networking 18.04






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Feb 15 at 10:42







      Shain Lafazan

















      asked Feb 15 at 9:48









      Shain LafazanShain Lafazan

      63




      63






















          1 Answer
          1






          active

          oldest

          votes


















          0














          I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.



          If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.






          share|improve this answer

























            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "89"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1118460%2fubuntu-server-18-04-bionic-network-settings-broken-after-power-cycle%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.



            If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.






            share|improve this answer






























              0














              I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.



              If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.






              share|improve this answer




























                0












                0








                0







                I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.



                If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.






                share|improve this answer















                I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.



                If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Feb 20 at 21:36

























                answered Feb 20 at 21:28









                Shain LafazanShain Lafazan

                63




                63






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Ask Ubuntu!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1118460%2fubuntu-server-18-04-bionic-network-settings-broken-after-power-cycle%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    How to reconfigure Docker Trusted Registry 2.x.x to use CEPH FS mount instead of NFS and other traditional...

                    is 'sed' thread safe

                    How to make a Squid Proxy server?