Initial support for multiple languages

[dkuku]

Closes #4
I prepared initial support for multiple languages.
This can be enabled in config by specifying the languages key.
It uses the translations provided by timex but can be switched to anything else.
I tested it with pl and I left the tests commented out because it need's to be enabled for the test to work.

[dbernheisel]

Looking good! However I don't want to expand using Timex due to #1. I can help by creating an IANA file parser to provide the timezone abbreviations, and maybe use/borrow ex_cldr for translating months.

What do you think?

[dkuku]

Sure, I just used timex because it was already installed. I have problems using it in projects due to the max version pinning.

…

On Sat, 3 Dec 2022, 17:12 David Bernheisel, ***@***.***> wrote: Looking good! However I don't want to expand using Timex due to #1 <#1>. I can help by creating an IANA file parser to provide the timezone abbreviations, and maybe use/borrow ex_cldr for translating months. What do you think? — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG4X46EPHZJ7GK352XWUJTWLN5RHANCNFSM6AAAAAASSV2MV4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[dkuku]

@dbernheisel I'm struggling to find in ex_cldr and the child libraries any functions that returns list of months and weekdays with the abbreviations. I may use some kind of hacks to get the values by formatting some fake date with only weekday or month but it's not the way to go

[dbernheisel]

@dkuku I tried this in Livebook, and it seems to work ok. You might conditionally check if the user has Cldr available, have the user pass in their Cldr backend, and date_time_parser use it to compile available translations. If the user doens't have Cldr in their system, I'm not sure I want to offer translations other than English.

---

ExCLDR Translation

Mix.install([
  {:ex_cldr_dates_times, "~> 2.0"}
])

Config

defmodule MyApp.Cldr do
  use Cldr,
    locales: ["en", "fr", "es"],
    default_locale: "en",
    providers: [Cldr.Number, Cldr.Calendar, Cldr.DateTime]
end

locales = MyApp.Cldr.known_locale_names()
anchor = ~D[2022-01-02]

for month <- 1..12 do
  date = Date.new!(anchor.year, month, anchor.day)

  trans =
    Enum.map(locales, fn l ->
      {l,
       [
         short: Cldr.DateTime.Formatter.month(date, 3, l, MyApp.Cldr),
         long: Cldr.DateTime.Formatter.month(date, 4, l, MyApp.Cldr)
       ]}
    end)

  {month, trans}
end

[
  {1,
   [
     en: [short: "Jan", long: "January"],
     es: [short: "ene", long: "enero"],
     fr: [short: "janv.", long: "janvier"]
   ]},
  {2,
   [
     en: [short: "Feb", long: "February"],
     es: [short: "feb", long: "febrero"],
     fr: [short: "févr.", long: "février"]
   ]},
  {3,
   [
     en: [short: "Mar", long: "March"],
     es: [short: "mar", long: "marzo"],
     fr: [short: "mars", long: "mars"]
   ]},
  {4,
   [
     en: [short: "Apr", long: "April"],
     es: [short: "abr", long: "abril"],
     fr: [short: "avr.", long: "avril"]
   ]},
  {5,
   [
     en: [short: "May", long: "May"],
     es: [short: "may", long: "mayo"],
     fr: [short: "mai", long: "mai"]
   ]},
  {6,
   [
     en: [short: "Jun", long: "June"],
     es: [short: "jun", long: "junio"],
     fr: [short: "juin", long: "juin"]
   ]},
  {7,
   [
     en: [short: "Jul", long: "July"],
     es: [short: "jul", long: "julio"],
     fr: [short: "juil.", long: "juillet"]
   ]},
  {8,
   [
     en: [short: "Aug", long: "August"],
     es: [short: "ago", long: "agosto"],
     fr: [short: "août", long: "août"]
   ]},
  {9,
   [
     en: [short: "Sep", long: "September"],
     es: [short: "sept", long: "septiembre"],
     fr: [short: "sept.", long: "septembre"]
   ]},
  {10,
   [
     en: [short: "Oct", long: "October"],
     es: [short: "oct", long: "octubre"],
     fr: [short: "oct.", long: "octobre"]
   ]},
  {11,
   [
     en: [short: "Nov", long: "November"],
     es: [short: "nov", long: "noviembre"],
     fr: [short: "nov.", long: "novembre"]
   ]},
  {12,
   [
     en: [short: "Dec", long: "December"],
     es: [short: "dic", long: "diciembre"],
     fr: [short: "déc.", long: "décembre"]
   ]}
]

for day <- 0..6 do
  date = Date.new!(anchor.year, anchor.month, anchor.day + day)

  trans =
    Enum.map(locales, fn l ->
      {l,
       [
         short: Cldr.DateTime.Formatter.day_name(date, 2, l, MyApp.Cldr),
         long: Cldr.DateTime.Formatter.day_name(date, 4, l, MyApp.Cldr)
       ]}
    end)

  {day, trans}
end

[
  {0,
   [
     en: [short: "Sun", long: "Sunday"],
     es: [short: "dom", long: "domingo"],
     fr: [short: "dim.", long: "dimanche"]
   ]},
  {1,
   [
     en: [short: "Mon", long: "Monday"],
     es: [short: "lun", long: "lunes"],
     fr: [short: "lun.", long: "lundi"]
   ]},
  {2,
   [
     en: [short: "Tue", long: "Tuesday"],
     es: [short: "mar", long: "martes"],
     fr: [short: "mar.", long: "mardi"]
   ]},
  {3,
   [
     en: [short: "Wed", long: "Wednesday"],
     es: [short: "mié", long: "miércoles"],
     fr: [short: "mer.", long: "mercredi"]
   ]},
  {4,
   [
     en: [short: "Thu", long: "Thursday"],
     es: [short: "jue", long: "jueves"],
     fr: [short: "jeu.", long: "jeudi"]
   ]},
  {5,
   [
     en: [short: "Fri", long: "Friday"],
     es: [short: "vie", long: "viernes"],
     fr: [short: "ven.", long: "vendredi"]
   ]},
  {6,
   [
     en: [short: "Sat", long: "Saturday"],
     es: [short: "sáb", long: "sábado"],
     fr: [short: "sam.", long: "samedi"]
   ]}
]

[dkuku]

Thanks-I was thinking about this but I was not sure about it, it felt a bit hacky. I'll try to do it tomorrow.

…

On Sat, 3 Dec 2022, 21:23 David Bernheisel, ***@***.***> wrote: @dkuku <https://github.com/dkuku> I tried this in Livebook, and it seems to work ok. You might conditionally check if the user has Cldr available, have the user pass in their Cldr backend, and date_time_parser use it to compile available translations. If the user doens't have Cldr in their system, I'm not sure I want to offer translations other than English. ------------------------------ ExCLDR Translation Mix.install([ {:ex_cldr_dates_times, "~> 2.0"} ]) Config defmodule MyApp.Cldr do use Cldr, locales: ["en", "fr", "es"], default_locale: "en", providers: [Cldr.Number, Cldr.Calendar, Cldr.DateTime] end locales = MyApp.Cldr.known_locale_names() for month <- 1..12 do trans = Enum.map(locales, fn l -> Cldr.DateTime.Formatter.month(Date.new!(2022, month, 1), 4, l, MyApp.Cldr) end) {month, trans} end [ {1, ["January", "enero", "janvier"]}, {2, ["February", "febrero", "février"]}, {3, ["March", "marzo", "mars"]}, {4, ["April", "abril", "avril"]}, {5, ["May", "mayo", "mai"]}, {6, ["June", "junio", "juin"]}, {7, ["July", "julio", "juillet"]}, {8, ["August", "agosto", "août"]}, {9, ["September", "septiembre", "septembre"]}, {10, ["October", "octubre", "octobre"]}, {11, ["November", "noviembre", "novembre"]}, {12, ["December", "diciembre", "décembre"]} ] — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG4X43CD6LUX5MZUUDTSTTWLO24RANCNFSM6AAAAAASSV2MV4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dkuku]

@dbernheisel It's working but I have problems to test it because the code is either compiled in one version or another.
I can switch between the implementations but I don't have any idea how to test it - easiest way would be to have 2 separate ci or refactor the code and have date_time_parser and date_time_parser_cldr packages.

[dkuku]

I'm thinking of some optimisation - instead of matching January and Jan we can try to just match all strings that are starting with Jan this should work out of the box with the current logic and reduce the amount of generated code.
I'm Polish and the month name can have multiple endings but it always starts the same. In cldr you have month and standalone_month which for english are the same but for other languages may not always be true - the abbrviation will work for all cases.

[dbernheisel]

@dkuku Once we have some tests added, let's get this merged! It's looking good.

[dkuku]

@dkuku Once we have some tests added, let's get this merged! It's looking good.

👍🏻 I can look into it during holidays.

[dkuku]

I can compile it and run the tests with mix tests - note the s at the end.
But I had to disable the warnings as errors check in mix.exs

[dbernheisel]

I have not forgotten about this :) I'm going to incorporate this in with some of my changes and get this going. I appreciate your work and your patience.

[dkuku]

That's why the 'allow edits from maintainers' checkbox is here - looking to see the next release :)

[dbernheisel]

Unfortunately there's an issue with this approach. In the combinators.ex.eex, we're using @month_map which gets compiled into the real combinators file by Nimble Parsec.

Where this is ok is in a function like vocal_month_to_numeric_month since that's called during runtime and the map is copied over into the final file, but the @months_map is also used by Nimble Parsec to construct the functions to know what to look for; this means that whenever a user adds :date_time_parser to their app, they'd have to re-compile combinators.ex.eex into combinators.ex in order to benefit from their Cldr.Backend which provides the translations.

Maybe this is ok, but it would mean that NimbleParsec would have to be a non-dev dependency and will be a 2-phase compilation again.

[dkuku]

Yes, you're right. But this would pe only needed if someone want's to configure the parser for custom language. By default it's not needed and the dependency would only be needed in dev envs for recompilation.

…

On Thu, 9 Mar 2023, 14:43 David Bernheisel, ***@***.***> wrote: Unfortunately there's an issue with this approach. In the combinators.ex.eex, we're using @month_map which gets compiled into the real combinators file by Nimble Parsec. Where this is ok is in a function like vocal_month_to_numeric_month since that's called during runtime and the map is copied over into the final file, but the @months_map is also used to determine which strings to look for and is inlined by NimbleParsec into functions; this means that whenever a user adds :date_time_parser to their app, they'd have to re-compile combinators.ex.eex into combinators.ex in order to benefit from their Cldr.Backend which provides the translations. Maybe this is ok, but it would mean that NimbleParsec would have to be a non-dev dependency and will be a 2-phase compilation again. — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG4X43ZJZPVG7ISLSBCG2LW3HT75ANCNFSM6AAAAAASSV2MV4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dbernheisel]

I'm going to close for now, mostly because of length of time open, but also there's an order-of-operations issue with this approach in that it requires cldr languages to be present during the build process of the library, and not the build process of the app using the library. We'll need to re-think the approach.