= Design notes for the NTPv5 protocol

== Why NTPv5? ==

Why an NTPv5?  Why not muddle through with NTPv4?  The principal
reason to avoid rollover cases in its time representation.  There is an
epoch turnover in 2036; it would be wise to have moved to full 64-bit
date stamps well before we have to deal with uncharted parts of the
software's behavior space.

NTPv4 is showing its age in other ways as well.  Its mechanism for
avoiding source loops does not play well with IPv6, and collisions
have been observed in the wild.

Since we need to deal with the 32-bit-counter issues anyway, the time
is right to design a protocol which (a) includes what we now know is
needed, and (b) is extensible for future uses.

== Packet metaprotocol

Every packet is led with a character '+'. This is chosen so that when
a packet analyzer for NTPv4 looks at the first byte, it sees 0x02B,
which unpacks to LI:00 VN:5 MODE:3.  The version 5, which is out of
band for an NTPv4 packet, will cause the packet to be rejected by
conforming NTPv4 implementations that do not handle v5.

The Mills reference version of NTP and all codebases descended from
it (including NTPSec) *do* recognize Version 5 as out of band
and will not pass such packets to a Version 4 protocol machine.

The leader byte is followed by an ASCII decimal numeric literal,
which is interpreted as a packet type.

Following the mode, the body of the packet is encoded in a strict
subset of JSON intended to avoid known edge cases described in
<<<Seriot2016>>>.  The outermost enclosing {} is required. Trailing
commas on elements are disallowed. Scientific notation is not used.
String literals may not contain the characters TAB (0x09), LF (0x0A),
CR (0x0D), "{", "}", or "|". The JSON MUST use only ASCII space (0x20)
as a whitespace character and MUST NOT use TAB, LF, CR or UTF-8
characters outside the ASCII range as whitespace.

The packet body MAY optionally be followed by an authentication
trailer. If present, the trailer consists of an ASCII "|" (0x7C) followed
by an octet string encoded as hex-digit pairs.

//FIXME: Add a description of how to compute the authentication trailer.

The packet MUST be terminated with an LF.

Transmitted packet sizes MUST be limited to 65,506 octets, so that the
largest possible NTPv5 packet and a trailing NUL will fit in one UDP
datagram.  To prevent DoS attacks, implementations MAY limit the
packet size they will receive to a lower value; however implementations
MUST accept packets of at least 512 octets.

In the remainder of this document, packet fields will be described as
keyword-value pairs. The keyword *is* explicit in the JSON, and the
mapping to on-the-wire representation is as implied in <<<RFC8259>>>.

Once the semantics of a packet field has been defined, it MAY be
omitted but MUST NOT change incompatibly. Protocol extensions MUST be
done with new fields.

Implementations MAY ignore fields they do not recognize, but SHOULD
make some effort to generate error notifications when this occurs.
Policy for rate-limiting such notifications is implementor's choice.

The Appendix discusses some paths not taken and the reasons they weren't.

== Packet types ==

We define five packet types:

[options="header"]
|===========================================================
| Type name        | NTPv4 equivalent | type | Description
| time poll        | mode 2 or 3      |  0   | request for time update
| time response    | mode 4           |  1   | time update
| control request  | mode 6           |  2   | request for server information
| control response | mode 6           |  3   | response with server information
| error            | KOD              |  4   | error notification
|===========================================================

Conforming implementations MUST respond to unknown packet types with
an error response.

== Time format and scale ==

A timestamp field is an MJD (Modified Julian Date) literal expressed
as a decimal integer with decimal fractional part.  The year may have
a leading minus sign, but is not expected to in normal operation.  The
epoch is midnight of November 17, 1858. Day fraction assumes the
TAI/UTC second.

An NTPv5 timestamp unambiguously represents leap seconds as values in
a monotonically increasing sequence.  This is a significant change
from the NTPv4 time representation, which is a UTC seconds counter
that stutters or skips on leap seconds.

Conforming servers MUST NOT transmit leap-smeared or otherwise
"adjusted" time; client implementations are expected to perform leap
smearing locally if at all.

== Time request packet

A time request packet MUST specify type 0. No other content is
required.  The packet body may be omitted.  (Thus, the minimal
request packet is "+0\n" where \n is a LF (ASCII 10),

Note: this corresponds to a minimal SNTP packet under RFC2030.

== Time response packet

A time response packet MUST specify type 1.

The following fields are retained from NTPv4:

"li":: Leap Indicator. Decimal numeric. Interpreted as in <<<RFC5905>>>.
      Optional, defaulting to 0 = no warning. Conforming
      implementations MUST report warning or unsynchronized
      status when appropriate.

"st":: Stratum. Decimal numeric. Interpreted as in RFC5905. Optional.

"in":: Poll. Decimal numeric. Interpreted as in RFC5905. Optional.

"pr":: Precision.   Decimal numeric. Interpreted as in RFC5905.

"de":: Root delay. Decimal numeric (seconds and fractional seconds).
       Interpreted as in RFC5905.

"di":: Root dispersion. Decimal numeric (seconds and fractional seconds).
       Interpreted as in RFC5905.

"rt":: Receive Timestamp. String, interpreted as timestamp format.

"tt":: Transmit Timestamp. String, interpreted as timestamp format.

"nc":: Cycle-detection ID. String, interpreted as an IPv4 or IPv6 address.
       Identifies the source host of the time update.

NTPv4 fields explicitly omitted from NTPv5 are version, reference ID,
origin timestamp, and reference timestamp.  The reference ID field
becomes the NTPv5 clock-type field.

//FIXME: Daniel gets to make the case why poll and stratum are useless.

Additional NTPv5 fields:

"id":: Request ID to be echoed in the response. Decimal numeric. Optional.

"lo":: Current leap-second offset from UTC. Decimal numeric. Optional.

"ct":: Clock type. String. Optional.  Identifies a clock source.
       Limited to 64 octets or less.

"au":: NTS authentication cookie for next exchange. String,
       interpreted as hex digit pairs.

== Control requests and responses.

Control request and responses MUST specify type 2 and 3 respectively.
A request is distinguished by the presence of a "params" field, a
response by the presence of a "result" field.

The following is an overly verbose partial mockup of a transaction
chain querying peer-stats.

[source, json]
----
{
   "id" : 1,
   "params" : {},
   "method" : "readstat"
}
{
   "id" : 1,
   "result" : {
      "answer" : {
         "associations" : [
            62414,
            62413,
            62408,
            62407,
            62406,
            62405,
            62402,
            62401,
            62400,
            62399,
            62398
         ]
      }
   }
}

{
   "id" : 2,
   "params" : {
      "association" : 62398
   },
   "method" : "readvar"
}
{
   "id" : 2,
   "result" : {
      "answer" : {
         "hmode" : 3,
         "filtdisp" : [
            14.68,
            1.5,
            2.36,
            3.45,
            4.75,
            5.19,
            6.19,
            7.12
         ],
         "keyid" : 0,
         "dstadr" : "127.0.0.1",
         "jitter" : 2.792031,
         "dstport" : 123,
         "rootdelay" : 0,
         "dispersion" : 8.528601,
         "flash" : 0,
         "filtoffset" : [
            -829.24,
            -831.68,
            -833.19,
            -832.72,
            -832.48,
            -831.32,
            -831.14,
            -830.83
         ],
         "reach" : 255,
         "mode" : 2,
         "rootdisp" : 0,
         "ppoll" : 6,
         "reftime" : 3757323811.47605,
         "delay" : 0,
         "offset" : -829.240892,
         "pmode" : 4,
         "srcadr" : "127.127.46.0",
         "precision" : -8,
         "headway" : 0,
         "hpoll" : 6,
         "rec" : 3757323811.5776,
         "xmt" : 3757323811.57759,
         "stratum" : 0,
         "srchost" : "GPSD(0)",
         "unreach" : 0,
         "srcport" : 123,
         "leap" : 0,
         "refid" : "GPSD",
         "filtdelay" : [
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0
         ]
      },
      "association" : 62398
   }
}


...
----

== Error notifications ==

An error notification has a type of 4 and has the following fields:

"id":: ID of the response to which this corresponsds. Optional.

"err":: Error code. String.

"msg":: Notification to human operator. String. Optional.

If none of these fields are present the JSON body may be omitted.
However, error notifications MUST be authenticated to prevent
destructive spoofing.

[options="header"]
|===========================================================
| Code | Description
| rate | Server telling client that the client's polling rate is excessive.
| down | Server telling client that the server cannot provide a time update.
| info | Informative message for operator.
|===========================================================

Other error types are reserved for expansion.

Historical note: These correspond to Kiss O'Death packets in NTPv4.

== Daniel weighs in

There aren't many deficiencies in NTPv4 which can't be fixed by adding
extension fields. A change big enough to make a version bump
worthwhile would incorporate at least most of the following:

1. Drop everything other than client/server mode. Replace mode 6 with
something that runs over HTTPS on the NTS-KE port.

2. Let client and server packets be formatted differently. Achieve
data minimization by just taking unnecessary fields out of client
packets altogether. (Achieved with JSON)

3. Forbid the use of the legacy MAC field, thus fixing the hairiness
around extension parsing. (Achieved with JSON)

4. Make NTS mandatory. In the NTPv5 packet format, the version, mode,
NTS unique identifier, and (in client packets) NTS cookie come first
in plaintext, then the whole rest of the packet is encrypted.

5. Ditch the useless poll, stratum, refid, and reference timestamp
fields. Given that all of the above are implemented, origin timestamp
also becomes redundant (NTS takes the place of its anti-spoofing
role).  (Achieved with JSON. but the poll and stratum may optionally
be reported.)

6. Represent timestamps as days, seconds, and fractions so that the
time can be represented unambiguously during leap seconds. Make the
day field 64 bits wide so that its range comfortable exceeds the
lifespan of the solar system.  (Achieved with JSON)

7. Don't implement leap smearing in the wire protocol (servers should
always report accurate, unsmeared time), but standardize a formula for
translating NTP time into smeared UNIX time seen by other
applications. (Specified.)

== Appendix: Paths not taken

We chose a JSON-based metaprotocol to achieve the following qualities:
(a) extensibility, (b) auditability by eyeball and simple tools, (c)
avoidance of endianness or fixed-length issues in numeric
representations.

We chose *not* to optimize for least possible size of packet. In NTPv4
this was is a choice that doesn't age well. Data and transaction
volumes in real-world NTP service are low enough that the overhead
of JSON with respect to packed binary is quite affordable.

(Note, however, that a minimal poll packet is 3 bytes rather than the
48 it would be in NTPv4.)

Two approaches we considered and rejected follow, with the
reasoning about why we rejected them.

=== NTPv4+

In this incremental approach, the NTP port number (123) would be
retained and the 48-byte v4 header would be preserved. New data
fields are passed in RFC7822 extension blocks.  The NTP version number
is not incremented; "v5" becomes a set of required extension blocks.

There can be a way to unambiguously detect v5 packets.  The stratum
field is 8 bits, but only the low five bits are used.  We can mark
v5 packets by setting the high three bits.

A difficulty with this approach is that some firewalls and routers are
known to silently discard RFC7822 extension blocks as a way of
preventing DoS attacks. However, by checking the mark described in the
previous paragraph, an ntpd can at least detect this.

=== NTPNG

In this approach, a new port number is allocated. The protocol
design is unconstrained except that it must carry the semantic
content of the v4 header minus the unused Reference Timestamp field.
The version field *is* incremented to 5.

the NTPNG payload should be structured like PNG, as a sequence of
self-describing chunks that can be retired and replaced as needed to
change payload semantics.

Though NTPNG is not constrained by the width of the v4 mode field,
the versionless semantics of a PNG-style chunk stream would confer a
desirable degree of flexibility.

The PNG standard can be found at https://www.w3.org/TR/PNG/

This space used to contain a description of a chunk system appropriate
for NTP, now omitted.

The principal difficulty with this approach is that getting all the
world's firewalls to pass through a new port is far from easy.  We
rejected it on these grounds.

== References
[bibiography]

- [[[RFC5905]]] https://tools.ietf.org/html/rfc5905[Network Time
  Protocol Version 4: Protocol and Algorithms Specification]

- [[[RFC8259]]] https://tools.ietf.org/html/rfc8259[The JavaScript
  Object Notation (JSON) Data Interchange Format]

- [[[Seriot2016]]] Seriot, Nicholas; "Parsing JSON is a Minefield"
  http://seriot.ch/parsing_json.php

// end

