Bug Report – NTP daemon not running on GLKVM firmware 1.8

Bug Report – NTP daemon not running on GLKVM firmware 1.8

Summary

On GLKVM devices (GL-RM1 / Comet) just updated with firmware v1.8, the NTP daemon (ntpd) is no longer started automatically despite the start file /etc/init.d/S49ntp.
This results in incorrect system time after boot, which breaks time-sensitive services such as WireGuard.

In firmware v1.7.x, ntpd was running by default and system time was synchronized correctly.


Impact

  • System clock remains incorrect after boot

  • WireGuard tunnels fail (handshakes rejected due to time skew)

  • Any TLS/time-dependent services may be affected

  • Device requires manual NTP sync to restore correct behavior


Observed Behavior (v1.8)

After boot:

[root@glkvm:~]# date
Thu Jan  1 00:01:42 UTC 1970
[root@glkvm:~]# ps -A | grep ntp
[root@glkvm:~]# 

→ No output (ntpd not running)

[root@glkvm:~]# ss -uapn | grep ':123'
[root@glkvm:~]# 

→ No output (no NTP socket)

System time is incorrect until manually synced.


Expected Behavior

As in firmware v1.7.x:

[root@glkvm:/etc]# ps aux | grep ntpd

Shows:

root     (...) /usr/sbin/ntpd -g

And system time is automatically synchronized at boot.


Workaround

Manual sync works:

[root@glkvm:~]# /usr/sbin/ntpd -g
  1. Performed local upgrade to 1.8.0
  2. Let it automatically reboot after update
  3. SSH in and confirmed time not synced, but NTPD daemon running.
[root@glkvm:~]# date
Wed Dec 31 19:02:35 -05 1969
[root@glkvm:~]# ps -A | grep -i ntp
 658 ?        00:00:00 ntpd
  1. Manually rebooted
reboot
  1. SSH in and confirmed time synced
[root@glkvm:~]# date
Sat Jan 31 12:30:36 -05 2026

  1. Had packet captures running when rebooted to confirm time sync
12:30:08.276469 94:83:c4:bb:17:1f > 64:62:66:21:c8:41, ethertype IPv4 (0x0800), length 90: (tos 0xb8, ttl 64, id 18757, offset 0, flags [DF], proto UDP (17), length 76)
    192.168.69.60.123 > 192.168.69.5.123: [udp sum ok] NTPv4, Client, length 48
	Leap indicator: clock unsynchronized (192), Stratum 0 (unspecified), poll 6 (64s), precision -20
	Root Delay: 0.000000, Root dispersion: 0.000076, Reference-ID: (unspec)
	  Reference Timestamp:  0.000000000
	  Originator Timestamp: 0.000000000
	  Receive Timestamp:    0.000000000
	  Transmit Timestamp:   2208988822.963163682 (1970-01-01T00:00:22Z)
	    Originator - Receive Timestamp:  0.000000000
	    Originator - Transmit Timestamp: 2208988822.963163682 (1970-01-01T00:00:22Z)
12:30:08.276697 64:62:66:21:c8:40 > 94:83:c4:bb:17:1f, ethertype IPv4 (0x0800), length 90: (tos 0xb8, ttl 64, id 22640, offset 0, flags [none], proto UDP (17), length 76)
    192.168.69.5.123 > 192.168.69.60.123: [bad udp cksum 0x0bdc -> 0xc700!] NTPv4, Server, length 48
	Leap indicator:  (0), Stratum 1 (primary reference), poll 6 (64s), precision -22
	Root Delay: 0.000000, Root dispersion: 0.001022, Reference-ID: GPS^@
	  Reference Timestamp:  3978869406.054694417 (2026-01-31T17:30:06Z)
	  Originator Timestamp: 2208988822.963163682 (1970-01-01T00:00:22Z)
	  Receive Timestamp:    3978869408.276523265 (2026-01-31T17:30:08Z)
	  Transmit Timestamp:   3978869408.276600296 (2026-01-31T17:30:08Z)
	    Originator - Receive Timestamp:  +1769880585.313359583
	    Originator - Transmit Timestamp: +1769880585.313436614

Summary

I have seen this behavior before where NTPD will not sync when it automatically reboots after firmware update. Once manually rebooted, it works correctly.

I rebooted it several times, but unfortunately ntpd did not start and the time was not synchronized. There was nothing in the logs.

Try using a custom NTP server. Example:

	echo "server 192.168.69.5 iburst minpoll 6 maxpoll 8" > /etc/ntp.conf

While investigating NTP synchronization issues at boot, I observed that ntpd is restarted multiple times by the led_event_controller process. Due to this behavior, ntpd does not reliably remain running after boot, and even when running, it does not synchronize time unless additional workarounds are applied.


Detailed observations

  1. Multiple restarts triggered by led_event_controller
    By analyzing the system logs, I observed that ntpd is repeatedly restarted by led_event_controller during the boot sequence.

  2. Final restart does not restart ntpd
    For reasons that are currently unclear, during the last call to

    /etc/init.d/S49ntp restart
    
    

    initiated by led_event_controller, ntpd is stopped but not restarted, leaving the daemon inactive at the end of the boot process.

  3. Workaround to keep ntpd running
    To prevent ntpd from being stopped permanently, I modified /etc/init.d/S49ntp by adding:

    exit 0
    
    

    as the first instruction of the restart logic.
    This change makes the restart operation inop and prevents repeated stop/start cycles. With this modification, ntpd remains running after boot.

  4. Time synchronization still fails despite -g / -G
    Even after ensuring that ntpd stays running, time synchronization does not occur at boot, despite starting ntpd with the -g (and even -G) options. The system clock remains at 1970.

  5. Final workaround: explicit time bootstrap
    Adding an explicit call to ntpdate as the first instruction of the start logic in /etc/init.d/S49ntp resolves the issue:

    • ntpdate immediately steps the clock once the network becomes available

    • ntpd can then operate normally and maintain synchronization

    With this change in place, time synchronization works reliably after boot.


Expected behavior

  • ntpd should not be restarted destructively multiple times during boot.

  • A final restart should always result in ntpd running.

  • Time synchronization should succeed at boot without requiring manual intervention.


Actual behavior

  • ntpd is restarted multiple times by led_event_controller.

  • The last restart leaves ntpd stopped.

  • Even when running, ntpd does not synchronize time at boot without an explicit ntpdate call.


Notes

This issue appears to be a combination of:

  • aggressive restart behavior in led_event_controller

  • ntpd being started before the network stack is fully ready

Have you ever modified ndp.conf?

Not a problem of led_controller. Otherwise, we would see plenty of user reports such problem.

Actually, NTPD is working very abnormally. It does not listen to changes in the network interface, and its time synchronization uses exponential backoff. Without manually restarting it, it simply won't sync time. This thing is poorly implemented, so we have to use the led_controller to manually restart it. Let me say it again: this thing is poorly designed. It can only function properly when paired with the led_controller

.

Hello,

nop.

So should I keep my workaround, or are you planning to fix this issue or replace ntpd with another daemon, such as chronyd?

Do you need more information to troubleshoot ? logs ?

Thank you,

Laurent

Confirmed fix.

On this Buildroot-based ARMv7 system, I replaced the bundled time service with a statically linked BusyBox with the ntpd applet, and time synchronization now works reliably.

BusyBox ntpd correctly syncs once the network is up and behaves as expected on embedded systems. No hacks or manual restarts needed.

[root@glkvm:~]# ls -l /bin/ntpd 
lrwxrwxrwx 1 root root 19 Feb  2 18:23 /bin/ntpd -> /bin/busybox-armv7l

[root@glkvm:~]# head -n10 /etc/init.d/S49ntp 
#! /bin/sh

NAME=ntpd
#DAEMON=/usr/sbin/$NAME
DAEMON=/bin/$NAME

[root@glkvm:~]# date
Mon Feb  2 18:32:25 UTC 2026

Since you are already using busybox, may be you can consider this solution.

Laurent

Can you try this firmware? We fixed the problem in led controller, and will restart NTPD only once when network became ready.

As we should keep the consistency across all products, we would not use the ntpd from busybox.(Not all SDK has the same busybox version.)

Same behavior I have observed with 1.8.0. NTP does not sync after firmware upgrade. Manually rebooting NTP starts to sync, but I disable the LED status daemon via a start script

        /bin/sh /etc/init.d/S23led stop
03:54:05.712712 94:83:c4:bb:17:1f > 64:62:66:21:c8:41, ethertype IPv4 (0x0800), length 90: (tos 0xb8, ttl 64, id 5683, offset 0, flags [DF], proto UDP (17), length 76)
    192.168.69.60.123 > 192.168.69.5.123: [udp sum ok] NTPv4, Client, length 48
	Leap indicator: clock unsynchronized (192), Stratum 0 (unspecified), poll 6 (64s), precision -20
	Root Delay: 0.000000, Root dispersion: 0.000122, Reference-ID: (unspec)
	  Reference Timestamp:  0.000000000
	  Originator Timestamp: 0.000000000
	  Receive Timestamp:    0.000000000
	  Transmit Timestamp:   2208988825.714827387 (1970-01-01T00:00:25Z)
	    Originator - Receive Timestamp:  0.000000000
	    Originator - Transmit Timestamp: 2208988825.714827387 (1970-01-01T00:00:25Z)
03:54:05.712941 64:62:66:21:c8:40 > 94:83:c4:bb:17:1f, ethertype IPv4 (0x0800), length 90: (tos 0xb8, ttl 64, id 64103, offset 0, flags [none], proto UDP (17), length 76)
    192.168.69.5.123 > 192.168.69.60.123: [bad udp cksum 0x0bdc -> 0xc228!] NTPv4, Server, length 48
	Leap indicator:  (0), Stratum 1 (primary reference), poll 6 (64s), precision -22
	Root Delay: 0.000000, Root dispersion: 0.001098, Reference-ID: GPS^@
	  Reference Timestamp:  3979097638.056174121 (2026-02-03T08:53:58Z)
	  Originator Timestamp: 2208988825.714827387 (1970-01-01T00:00:25Z)
	  Receive Timestamp:    3979097645.712777848 (2026-02-03T08:54:05Z)
	  Transmit Timestamp:   3979097645.712842337 (2026-02-03T08:54:05Z)
	    Originator - Receive Timestamp:  +1770108819.997950460
	    Originator - Transmit Timestamp: +1770108819.998014949

Hello,

with this firmware, the NTP daemon is launched by rcS and then restarted only once by the led controller.

The device is now synchronized, so I think it’s ok .

Laurent

1 Like