Skip to content

encode: don't shift local datetimes/dates/times to UTC#488

Merged
arp242 merged 1 commit into
BurntSushi:masterfrom
sarathfrancis90:fix-local-time-encode-utc-shift
Jun 27, 2026
Merged

encode: don't shift local datetimes/dates/times to UTC#488
arp242 merged 1 commit into
BurntSushi:masterfrom
sarathfrancis90:fix-local-time-encode-utc-shift

Conversation

@sarathfrancis90

Copy link
Copy Markdown
Contributor

I noticed that local datetimes, dates, and times don't survive a round trip on any machine that isn't in UTC: decoding then re-encoding shifts the value by the local timezone offset.

var v map[string]any
toml.Decode("t = 07:32:00", &v)

var buf bytes.Buffer
toml.NewEncoder(&buf).Encode(v)
// buf is "t = 11:32:00\n" on a UTC-4 machine

A local time of 07:32:00 comes back as 11:32:00, and a near-midnight local datetime can even roll over to a different day. These types have no offset, so their value is the literal wall-clock time — the decoder already stores it that way via time.ParseInLocation, but the encoder ran v.In(time.UTC) before formatting, which moved the wall clock.

The fix formats the wall-clock value directly. I also updated the toml-test helper's parseTime to build local times with time.ParseInLocation so it constructs them the same way the decoder does (the previous time.Parse(...).In(l) and the encoder's UTC shift were cancelling each other out, which is why the conformance suite didn't catch this).

Tests pass under several timezones (UTC, America/New_York, Pacific/Kiritimati); added a round-trip test for the local types. The existing ossfuzz round-trip harness didn't surface this because it only re-decodes the output, it doesn't compare values.

@arp242

arp242 commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

They're not set to UTC, they're set to internal.LocalDatetime, internal.LocalDate, and internal.LocalTime. Go doesn't really have a good way to represent "time without a timezone attached", and AFAIK this is the best we can do.

I'm not sure if setting it to the computer's local timezone is necessarily better.

We should maybe export these though. And document it better. I agree all of this is confusing and annoying (it also came up in the pq SQL driver recently).

@arp242

arp242 commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

The main issue here by the way is being able to distinguish between:

datetime       = 2021-05-06 15:16:17+08:00
datetime-local = 2021-05-06 15:16:17
date-local     = 2021-05-06
time-local     = 15:16:17

If we'd just parse it to time.Local, that would get lost. And e.g. date-local = 2021-05-06 no longer roundtrips to a date, but a datetime.

I'm not saying we can't improve something here, maybe, but I'm not sure this PR as it is now is really an improvement. Open to suggestions, but round-tripping the format is more useful IMO.

@arp242 arp242 closed this Jun 14, 2026
@arp242 arp242 reopened this Jun 14, 2026
@arp242

arp242 commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

Oops, didn't mean to close.

@sarathfrancis90

Copy link
Copy Markdown
Contributor Author

Right — and Iʼm not changing any of that. Decoding is untouched, so values still come back tagged with internal.LocalDatetime/LocalDate/LocalTime, and the encoder still selects the format from v.Location() exactly as before. date-local still encodes as a date and time-local as a time — the distinction is preserved.

The only change is the format call itself. Those sentinels are FixedZone("…-local", localOffset), so a parsed 15:16:17 is wall-clock 15:16:17 at localOffset. The old code did v.In(time.UTC).Format(...), which re-projects that instant onto UTC before printing and so shifts the printed value by localOffset — on a UTC-4 box time-local = 15:16:17 re-encodes to 19:16:17. v.Format(...) prints the wall clock in the zone it was parsed in, which is what round-trips.

So this isnʼt "parse to time.Local" — it keeps the existing local-zone design and just stops shifting the value on the way out. The Decoder.Timezone configurability in that TODO is orthogonal; this is only the double-offset on encode. Happy to expand the comment or add a round-trip case in toml-test if youʼd prefer.

@arp242

arp242 commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

Okay, thanks – maybe I misunderstood the diff then. I'll have a closer look later.

@arp242 arp242 force-pushed the fix-local-time-encode-utc-shift branch from 9dde78f to dff0d5a Compare June 27, 2026 13:32
A local datetime, date, or time carries no timezone offset, so its value
is the literal wall-clock time. The decoder stores it as such using
time.ParseInLocation, but the encoder formatted it with v.In(time.UTC)
first. On any machine not in UTC this shifted the value by the local
offset, so Decode -> Encode -> Decode changed the time (e.g. a local
time of 07:32:00 became 11:32:00 in UTC-4, and a near-midnight local
datetime could even roll over to a different day).

Format the wall-clock value directly instead. Also fix the toml-test
helper's parseTime to build local times with time.ParseInLocation so it
matches how the decoder constructs them.
@arp242 arp242 force-pushed the fix-local-time-encode-utc-shift branch from dff0d5a to 24ca3d8 Compare June 27, 2026 16:36
@arp242 arp242 merged commit c6d720d into BurntSushi:master Jun 27, 2026
8 checks passed
@arp242

arp242 commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants