Ubuntu server 18.04 (bionic) network settings broken after power cycle
Networking was fine before reboot. After reboot, the system and all network settings are incorrect.
Notes:
- Ubuntu Server 18.04 (Bionic)
- Canonical livepatch was set up a few days ago.
- The new network thingy, netplan, doesn't seem to be installed (however there is a file
/etc/netplan/50-cloud-init.yaml
) - The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).
- The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:
/etc/resolv.conf
is symlinked to/run/systemd/resolve/stub-resolv.conf
file. When I update it manually withnameserver 192.168.1.1
it works.
ping 192.168.1.1
connect: Network is unreachable
ping google.com
ping: google.com: System error
dig google.com
connection timed out ...
...
ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
ipconfig enp7s0 up
ping 192.168.1.1
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
nslookup google.com
Server: 192.168.1.1
Address: 192.168.1.1#53
Non-authoritative answer:
Name google.com
Address 172.217.164.110
...
ping google.com
ping: google.com: System error # ... still network issue, most tools do not resolve hosts
systemd-resolve google.com
google.com: 216.58.195.238
-- Information acquired via protocol DNS in 3.0494s.
-- Data is authenticated: no
I've also tried configuring /etc/resolv.conf
/etc/nsswitch.conf
to no avail.
And on reboot, it seems that the network managers, maybe systemd-networkd
and systemd-resolved
reset some of the /etc files to settings that don't work.
Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.
networking 18.04
add a comment |
Networking was fine before reboot. After reboot, the system and all network settings are incorrect.
Notes:
- Ubuntu Server 18.04 (Bionic)
- Canonical livepatch was set up a few days ago.
- The new network thingy, netplan, doesn't seem to be installed (however there is a file
/etc/netplan/50-cloud-init.yaml
) - The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).
- The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:
/etc/resolv.conf
is symlinked to/run/systemd/resolve/stub-resolv.conf
file. When I update it manually withnameserver 192.168.1.1
it works.
ping 192.168.1.1
connect: Network is unreachable
ping google.com
ping: google.com: System error
dig google.com
connection timed out ...
...
ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
ipconfig enp7s0 up
ping 192.168.1.1
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
nslookup google.com
Server: 192.168.1.1
Address: 192.168.1.1#53
Non-authoritative answer:
Name google.com
Address 172.217.164.110
...
ping google.com
ping: google.com: System error # ... still network issue, most tools do not resolve hosts
systemd-resolve google.com
google.com: 216.58.195.238
-- Information acquired via protocol DNS in 3.0494s.
-- Data is authenticated: no
I've also tried configuring /etc/resolv.conf
/etc/nsswitch.conf
to no avail.
And on reboot, it seems that the network managers, maybe systemd-networkd
and systemd-resolved
reset some of the /etc files to settings that don't work.
Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.
networking 18.04
add a comment |
Networking was fine before reboot. After reboot, the system and all network settings are incorrect.
Notes:
- Ubuntu Server 18.04 (Bionic)
- Canonical livepatch was set up a few days ago.
- The new network thingy, netplan, doesn't seem to be installed (however there is a file
/etc/netplan/50-cloud-init.yaml
) - The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).
- The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:
/etc/resolv.conf
is symlinked to/run/systemd/resolve/stub-resolv.conf
file. When I update it manually withnameserver 192.168.1.1
it works.
ping 192.168.1.1
connect: Network is unreachable
ping google.com
ping: google.com: System error
dig google.com
connection timed out ...
...
ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
ipconfig enp7s0 up
ping 192.168.1.1
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
nslookup google.com
Server: 192.168.1.1
Address: 192.168.1.1#53
Non-authoritative answer:
Name google.com
Address 172.217.164.110
...
ping google.com
ping: google.com: System error # ... still network issue, most tools do not resolve hosts
systemd-resolve google.com
google.com: 216.58.195.238
-- Information acquired via protocol DNS in 3.0494s.
-- Data is authenticated: no
I've also tried configuring /etc/resolv.conf
/etc/nsswitch.conf
to no avail.
And on reboot, it seems that the network managers, maybe systemd-networkd
and systemd-resolved
reset some of the /etc files to settings that don't work.
Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.
networking 18.04
Networking was fine before reboot. After reboot, the system and all network settings are incorrect.
Notes:
- Ubuntu Server 18.04 (Bionic)
- Canonical livepatch was set up a few days ago.
- The new network thingy, netplan, doesn't seem to be installed (however there is a file
/etc/netplan/50-cloud-init.yaml
) - The machine does seem to acquire a DHCP lease with the router (will connect over ethernet, with ip address 192.168.1.29).
- The enp7s0 interface (which I believe is the desired connecting device) is DOWN on boot. Manually bringing up the enp7s0 interface allows connection to router:
/etc/resolv.conf
is symlinked to/run/systemd/resolve/stub-resolv.conf
file. When I update it manually withnameserver 192.168.1.1
it works.
ping 192.168.1.1
connect: Network is unreachable
ping google.com
ping: google.com: System error
dig google.com
connection timed out ...
...
ipconfig enp7s0 192.168.1.29 netmask 255.255.255.0 broadcast 192.168.1.255
ipconfig enp7s0 up
ping 192.168.1.1
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
64 bytes from 192.168.1.1: ...
nslookup google.com
Server: 192.168.1.1
Address: 192.168.1.1#53
Non-authoritative answer:
Name google.com
Address 172.217.164.110
...
ping google.com
ping: google.com: System error # ... still network issue, most tools do not resolve hosts
systemd-resolve google.com
google.com: 216.58.195.238
-- Information acquired via protocol DNS in 3.0494s.
-- Data is authenticated: no
I've also tried configuring /etc/resolv.conf
/etc/nsswitch.conf
to no avail.
And on reboot, it seems that the network managers, maybe systemd-networkd
and systemd-resolved
reset some of the /etc files to settings that don't work.
Does anyone have a thought for what happened? I can post logs or whatever can help diagnose.
networking 18.04
networking 18.04
edited Feb 15 at 10:42
Shain Lafazan
asked Feb 15 at 9:48
Shain LafazanShain Lafazan
63
63
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.
If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1118460%2fubuntu-server-18-04-bionic-network-settings-broken-after-power-cycle%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.
If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.
add a comment |
I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.
If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.
add a comment |
I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.
If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.
I investigated the issue on my own -- it seems that a Canonical Livepatch failed on my machine (I haven't found the root cause yet). When it failed, it had only partially applied updates to cloud-init, and while netplan was not installed, many network settings had been changed to assume it was. This left multiple network configurations in a partially migrated state. While I was able to restore some network connectivity by manually changing settings, the work involved with finding and configuring them was astronomical, and many changes were not persistent without a boot script. The most reasonable solution I found was to reinstall the operating system and disable Canonical Livepatch.
If it is indeed a network configuration patch for 18.04 that caused this, @Canonical should remember that 18.04 is designated LTS -- the live patching should NOT under any circumstances apply updates that risk loss of network settings for LTS servers. If this is the case, it is extremely careless behavior. If anyone from Canonical reads this, I would appreciate any comment from someone with more knowledge of the Livepatch internals to confirm and assess root cause.
edited Feb 20 at 21:36
answered Feb 20 at 21:28
Shain LafazanShain Lafazan
63
63
add a comment |
add a comment |
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1118460%2fubuntu-server-18-04-bionic-network-settings-broken-after-power-cycle%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown