Announcement

Collapse
No announcement yet.

Kernel takes more than a minute to boot

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [SOLVED] Kernel takes more than a minute to boot

    System Specs:
    CPU: Intel i9-990k
    Motherboard: ROG Strix-Z390-F Gaming
    RAM: 32GB
    Graphics: ASUS DUAL-RTX2080S-O8G-EVO
    Related OS: Kubuntu 20.04
    Relevant storage:
    Boot - Samsung 970 EVO 500GB (/boot/efi)
    Linux: SanDisk SD7SB3Q-128G-1006 SSD (/root and swap)
    Home: ADATA SP610 SSD (/home)
    Multiple other HDDs, and SSDs, mounted as either ntfs-3g, or ext4

    Not quite sure if I'm putting this in the right place. At first, Kubuntu booted up fast, with a kernel boot time of just seconds, after a clean install. After some time, and updates, the kernel takes more than 1 minute to boot.

    The only thing I really ried to this point was disabling fstrim.service, and enabling fstrim.timer, because initially, systemd-analyze blame said that fstrim.service was the culprit. Changing this removed fstrim from systemd-analyze blame, but had no impact on boot time.

    Output of systemd-analyze:
    Code:
    Startup finished in 28.028s (firmware) + 6.479s (loader) + 1min 8.375s (kernel) + 12.455s (userspace) = 1min 55.339s 
    graphical.target reached after 12.449s in userspace
    Output of systemd-analyze blame:
    Code:
    7.630s NetworkManager-wait-online.service                                                       
    3.491s media-jason-Users.mount                                                                  
    3.376s upower.service                                                                           
    3.163s media-jason-Virtual.mount                                                                
    3.099s media-jason-ROMs.mount                                                                   
    2.399s media-jason-VM.mount                                                                     
    1.556s snap-snapd-12883.mount                                                                   
    1.551s snap-wine\x2dplatform\x2druntime-241.mount                                               
    1.440s snap-snapd-12398.mount                                                                   
    1.437s snap-spotify-53.mount                                                                    
    1.233s snap-notepad\x2dplus\x2dplus-292.mount                                                   
    1.163s snap-wine\x2dplatform\x2d5\x2dstable-16.mount                                            
    1.142s snap-spotify-52.mount                                                                    
    1.083s dev-sdb2.device                                                                          
    1.076s snap-keepassxc-1309.mount                                                                
     998ms snap-wine\x2dplatform\x2d6\x2dstable-8.mount                                             
     975ms snap-wine\x2dplatform\x2d6\x2dstable-5.mount                                             
     788ms snapd.service                                                                            
     785ms snap-wine\x2dplatform\x2druntime-229.mount                                               
     738ms dev-loop1.device                                                                         
     679ms dev-loop2.device                                                                         
     630ms snap-notepad\x2dplus\x2dplus-285.mount                                                   
     626ms dev-loop0.device                                                                         
     583ms snap-gnome\x2d3\x2d28\x2d1804-161.mount                                                  
     581ms dev-loop3.device                                                                         
     569ms systemd-logind.service                                                                   
     547ms dev-loop4.device                                                                         
     544ms dev-loop5.device                                                                         
     493ms dev-loop6.device                                                                         
     442ms networkd-dispatcher.service                                                              
     409ms udisks2.service                                                                          
     373ms dev-loop10.device                                                                        
     367ms systemd-journal-flush.service                                                            
     338ms dev-loop7.device                                                                         
     331ms snap-wine\x2dplatform\x2d5\x2dstable-13.mount                                            
     325ms dev-loop8.device                                                                         
     312ms snap-core18-2074.mount                                                                   
     303ms snap-gtk\x2dcommon\x2dthemes-1515.mount                                                  
     299ms snap-keepassxc-1522.mount                                                                
     272ms snapd.seeded.service                                                                     
     270ms dev-loop15.device                                                                        
     252ms snap-gtk\x2dcommon\x2dthemes-1514.mount                                                  
     242ms dev-loop9.device                                                                         
     236ms accounts-daemon.service                                                                  
     215ms systemd-fsck@dev-disk-by\x2duuid-bbe9141e\x2d6ac6\x2d4f98\x2dba47\x2d95c633972c12.service
     214ms dev-loop16.device                                                                        
     193ms dev-loop14.device                                                                        
     182ms dev-loop17.device                                                                        
     181ms avahi-daemon.service                                                                     
     179ms bluetooth.service                                                                        
     178ms nmbd.service                                                                             
     176ms snap-gnome\x2d3\x2d28\x2d1804-145.mount                                                  
     176ms NetworkManager.service                                                                   
     169ms dev-loop11.device                                                                        
     167ms polkit.service                                                                           
     165ms systemd-fsck@dev-disk-by\x2dlabel-Linux\x2dGames.service                                 
     152ms switcheroo-control.service                                                               
     148ms thermald.service                                                                         
     145ms wpa_supplicant.service                                                                   
     138ms dev-loop12.device                                                                        
     127ms snap-core18-2128.mount                                                                   
     126ms teamviewerd.service                                                                      
     115ms ua-messaging.service                                                                     
     107ms gpu-manager.service                                                                      
     104ms user@1000.service                                                                        
      97ms systemd-resolved.service                                                                 
      94ms dev-loop13.device                                                                        
      92ms virtualbox.service                                                                       
      88ms secureboot-db.service                                                                    
      82ms systemd-udevd.service                                                                    
      80ms ModemManager.service                                                                     
      77ms apport.service                                                                           
      73ms systemd-journald.service                                                                 
      71ms systemd-udev-trigger.service                                                             
      71ms smbd.service                                                                             
      68ms grub-common.service                                                                      
      65ms rsyslog.service                                                                          
      56ms e2scrub_reap.service                                                                     
      44ms systemd-timesyncd.service                                                                
      42ms dev-disk-by\x2duuid-013e5ca3\x2d8edf\x2d438c\x2d95b6\x2d4e80c04e6954.swap                
      41ms keyboard-setup.service                                                                   
      40ms apparmor.service                                                                         
      40ms lm-sensors.service                                                                       
      39ms systemd-sysctl.service                                                                   
      38ms ssh.service                                                                              
      35ms systemd-rfkill.service                                                                   
      31ms systemd-modules-load.service                                                             
      30ms systemd-tmpfiles-clean.service                                                           
      28ms systemd-fsck@dev-disk-by\x2duuid-4241\x2d438D.service                                    
      26ms dev-loop18.device                                                                        
      22ms snapd.apparmor.service                                                                   
      22ms systemd-tmpfiles-setup.service                                                           
      21ms plymouth-quit.service                                                                    
      20ms nvidia-persistenced.service                                                              
      20ms media-jason-Linux\x2dGames.mount                                                         
      17ms grub-initrd-fallback.service                                                             
      15ms pppd-dns.service                                                                         
      13ms systemd-sysusers.service                                                                 
      13ms kerneloops.service                                                                       
      12ms plymouth-start.service                                                                   
      12ms systemd-tmpfiles-setup-dev.service                                                       
      11ms plymouth-read-write.service                                                              
      11ms systemd-random-seed.service                                                              
      10ms alsa-restore.service                                                                     
       9ms systemd-remount-fs.service                                                               
       8ms boot-efi.mount                                                                           
       8ms systemd-user-sessions.service                                                            
       8ms user-runtime-dir@1000.service                                                            
       7ms media-jason-ROM\x2dSaves.mount                                                           
       7ms home.mount                                                                               
       6ms dev-loop19.device                                                                        
       6ms systemd-update-utmp.service                                                              
       6ms console-setup.service                                                                    
       6ms openvpn.service                                                                          
       5ms setvtrgb.service                                                                         
       5ms dev-hugepages.mount                                                                      
       5ms dev-mqueue.mount                                                                         
       5ms systemd-update-utmp-runlevel.service                                                     
       5ms sys-kernel-debug.mount                                                                   
       5ms ufw.service                                                                              
       4ms sys-kernel-tracing.mount                                                                 
       3ms rc-local.service                                                                         
       3ms kmod-static-nodes.service                                                                
       2ms rtkit-daemon.service                                                                     
       2ms sddm.service                                                                             
       1ms sys-fs-fuse-connections.mount                                                            
       1ms sys-kernel-config.mount                                                                  
     438us snapd.socket
    Output of systemd-analyze critical-chain:
    Code:
    The time when unit became active or started is printed after the "@" character.
    The time the unit took to start is printed after the "+" character.
    
    graphical.target @12.449s
    └─multi-user.target @12.449s
      └─smbd.service @12.377s +71ms
        └─nmbd.service @12.198s +178ms
          └─network-online.target @12.197s
            └─NetworkManager-wait-online.service @4.566s +7.630s
              └─NetworkManager.service @4.388s +176ms
                └─dbus.service @4.387s
                  └─basic.target @4.378s
                    └─sockets.target @4.378s
                      └─snapd.socket @4.378s +438us
                        └─sysinit.target @4.374s
                          └─systemd-timesyncd.service @4.329s +44ms
                            └─systemd-tmpfiles-setup.service @4.306s +22ms
                              └─local-fs.target @4.303s
                                └─media-jason-Users.mount @812ms +3.491s
                                  └─dev-sdc1.device @799ms
    Any help would be greatly appreciated.

    #2
    "Over a minute" suggests a systemd timeout, which default to 90 s. These can be a nuisance to track down, but the timeout length can be changed from the default.

    Have you turned on boot messages to see what's happening? (To do so, edit /etc/default/grub to remove the words "quiet splash" from the variable settings where they appear, then run sudo update-grub in a konsole.)

    Often it's a network timeout accessing a device on a system that's not on or not there. Starting Kubuntu with an empty session might help.
    Regards, John Little

    Comment


      #3
      Code:
      Startup finished in 28.028s (firmware)
      This may be the place to look, though it won't be obvious what the real culprit is.
      This 28 seconds is the time it takes to get past the bios POST, before grub or anything else, so this very likely is not OS related at all.

      Do you have a USB backup drive or similar connected?
      I ask as I myself have a 'long' boot time, with moist of it being similar toy your output - a very long time to get past the bios, sometimes well over 30 seconds. After messing with various settings, I sort of gave up, figuring the HP just had a sh***ty bios. Yesterday, I accidentally unplugged my external USB backup drive, and neglected to plug it back in. My boot time in the firmware section dropped to just over 10 seconds, which is nearly 1/3 of the previous time. The rest of the boot seems faster, but I don't remember the overall time it used to be with the drive connected.





      Your Snaps do add some time to logins, but do note that the times you see in systemd-analyze blame are not cumulative, many things are loaded in parallel, so remove one tiem tjhat takes 2 seconds or something won't speed up boot time by those 2 seconds.

      But you do have a lot of things being mounted (.mount and loop devices, not just from snaps.)

      the NetworkManager-wait-online.service is worth investigating, this can hold up other things, and is quite common for some people to notice and learn how to disable. A few seconds is a few seconds

      Comment


        #4
        replying in order. jlittle, I have not tried with the splash turned off, and I may try this, but I always start with an empty session. (This is one of the first things I change in a new install, because I love fresh sessions)

        Firmware startup isn't something I'm worried about. I know what causes that, and it's the same delay for Windows. I have 1 bad physical USB port, that I never use, and another that's always reported, but isn't physically connected to the system. The board has a USB-C connector that the case didn't have a plug for, and the other was burned out by a power surge. Pretty sure the 30-ish second POST delay is caused by these, and I ignore this problem.

        Literally the only time I'm worried about dropping is the kernel boot time, which is just over a minute. I get rid of that slow down, and my boot times will be right in-line with Windows 10. This system dual-boots with Windows 10. Once that boot gets past the 30-ish seconds worth of POST delays, it's up, and waiting for me to login in around 10 seconds.

        I have a fat32 external drive that is always connected to the system. I call it my travel drive. It stays connected to this PC, unless I go somewhere with my laptop, as it has files I use on both. I could try disconnecting that on my next boot.

        Hard drive mounts are all noted in fstab, which has been checked several times, to make sure those are set correctly, and that the noauto option is used on any drive that might not be attached on boot (travel drive, for example)

        I use grub as my OS Choice menu. When I choose Windows, normal boot, no issues. When I choose ubuntu, it goes back to my POST splash, and I don't see the Kubuntu splash, until a full minute has passed. Once I see the Kubuntu splash, boot takes only seconds.

        Comment


          #5
          Look at the large number of snaps, and their respective mounts.

          look a the first things unrelated to snaps:
          Code:
          7.630s NetworkManager-wait-online.service                                                       
          3.491s media-jason-Users.mount                                                                  
          3.376s upower.service                                                                           
          3.163s media-jason-Virtual.mount                                                                
          3.099s media-jason-ROMs.mount                                                                   
          2.399s media-jason-VM.mount
          These may be causing slowdown/waits as other startup services wait for these to finish

          You do have an fairly large number of snap applications + their respective loops/mounts, and dependency loops/mounts. This could be a factor as well. Well, very likely is the factor. Count the number of loop and mount items in your startup.

          So, imo you might want to look at disabling NetworkManager-wait-online.service, this seems to be a common item that can slow down the loading of other services down the line that are waiting for the network to load. Some people (not all!) see good improvements without it.

          and then try disabling or removing some of your snaps, including the ones you won't see in a GUI software center.

          Comment


            #6
            I also realized I missed clarifying my point about drives and a slow POST. 30 seconds is a LONG time for this process, and my removing the USB backup drive not only sped that part up by 2/3, it also reduced the overall boot time from ~ 1 minute to under 25 seconds. It may not be anything major, but examining the drives(s) and partitions might also be worth looking at if Snaps are not the main cause for your slow boot.

            Comment


              #7
              Here's what I've done, and the results there-of:

              commented out the "no-splash" line in /etc/default/grub, and ran sudo update-grub. When I watched, I saw my custom NTFS mounts take an abnormally long time to mount. The 2TB "Users" partition took 20 minutes alone. I set all of those to "noauto" in fstab, and it stopped that issue, but the txt during boot flies by, so I have no time to read anything. There a way to do a one-time output of ALL of that text to a text file of my choosing, so I can peruse through it easier? (I did see a timeout on a device, but it was gone, before I could read more than that.)

              removed the snap packages notepad-plus-plus, and spotify. I use keepassxc a lot, so keeping that. Here's the remaining list. Not sure about those Wine packages, because I don't know if keepassxc uses one of them. (I'm assuming they're for some snaps) What can I safely remove from here?
              Code:
              core18
              gnome-3-28-1804
              gtk-common-themes
              keepassxc
              snapd
              wine-platform-5-stable  5.0.3
              wine-platform-6-stable  6.0.1
              wine-platform-runtime   v1.0
              Since NetworkManager-wait-online.service was mentioned, I should point out that I did not disable it, because in certain circumstances, I need access to the computer from the network, to issue commands at the login screen, but not before that point. (usually to change to the display on my bed, since I'm disabled) If I can disable this, and still issue commands via ssh, once i reach the GUI Login screen, then let me know, and I will try this step.

              You mentioned loop devices. I see those in the blame list, but not sure what they are. I do have a xbox controller driver, which allows me to use my xbox one controller, with it's own wireless dongle, as opposed to plugging it in. I also have a few different bluetooth devices.

              The above changes served only to shorten the userspace boot time from 12.4s to about 9.8 seconds. The kernel boot time has not changed.

              Here's the breakdown of boot times, and where the issue is)
              * POST (ROG Splash) (30s average, don't care about this)
              * GRUB Boot menu (however long it takes me to choose Linux)
              * Kernel boot (ROG Splash, 1m8s)
              * userspace boot (Kubuntu splash, 9.8s, don't care about this)

              Nothing seems to be shortening the kernel boot time, which I've seen as low as 2-3s on other posts I've found here about slow boot times. This userspace stuff is helping me make normal boot times quicker, so thanks for those tips, but this is abnormal, and appears to be happening earlier than the stuff you're suggesting... unless I'm misunderstanding? (I have a good grasp of Linux, but am still kinda new.)

              oh... the POST has been this way, since I installed the board to a case with no USB-C connector included. I might turn off the splash screen there, to see what it's doing behind closed doors. I've also unplugged the external drive. will update you on my next reboot to Linux.

              Comment


                #8
                Since NetworkManager-wait-online.service was mentioned, I should point out that I did not disable it, because in certain circumstances, I need access to the computer from the network, to issue commands at the login screen, but not before that point. (usually to change to the display on my bed, since I'm disabled) If I can disable this, and still issue commands via ssh, once i reach the GUI Login screen, then let me know, and I will try this step.
                This won't effect that, it just stops the boot process from waiting for a network connection before proceeding. Network Manager is still going to load things, the rest of the system just won't be waiting around for it to get it's butt into gear. In any case, it is the same sort of step to re-enable it.

                Comment


                  #9
                  This problem is resolved. when I disabled the service, and restarted, I used my phone to take pictures of some of the pauses in text.

                  Your USB backup drive theory was partially correct. I use a 28 port hub, to connect everything, except my VR headset, keyboard, and mouse. Turns out that some of the ports there have gone bad. Moving everything out of the bad ports resolved the issue with both the Linux boot time, and the POST time. It also tells me that what I thought was a burned out USB port on my PC was actually an item plugged into a bad port on the hub. THANK YOU!

                  Code:
                  Startup finished in 15.521s (firmware) + 7.520s (loader) + 4.983s (kernel) + 12.597s (userspace) = 40.622s 
                  graphical.target reached after 12.591s in userspace

                  Comment


                    #10
                    Originally posted by jasoncollege24 View Post


                    Code:
                    Startup finished in 15.521s (firmware) + 7.520s (loader) + 4.983s (kernel) + 12.597s (userspace) = 40.622s 
                    graphical.target reached after 12.591s in userspace
                    Zooom!

                    Sweet

                    Comment

                    Working...
                    X