• Squish __ftsc_date bug / JAM bug

    From Oli@2:280/464.47 to andrew clarke on Sunday, February 14, 2021 11:28:30
    andrew wrote (2021-02-10):

    10 Feb 21 09:56, you wrote to Kai Richter:

    It's just a simple bug ...

    I suspect the patch below fixes this, but I've not tested it.

    It's also necessary to verify it doesn't break when rescanning *.MSG and JAM bases.

    I tested it with a JAM base and it still works. But it turned out a rescan from
    a JAM base also modifies time stamps:

    Original:
    Date and time : 09 Feb 21 06:31:33
    Date and time : 09 Feb 21 05:08:19
    Date and time : 07 Feb 21 18:49:35
    Date and time : 10 Feb 21 03:55:02
    Date and time : 10 Feb 21 08:48:36
    Date and time : 10 Feb 21 12:23:00
    Date and time : 10 Feb 21 13:11:33

    After rescan from JAM:
    Date and time : 09 Feb 21 06:31:32
    Date and time : 09 Feb 21 05:08:18
    Date and time : 07 Feb 21 18:49:34
    Date and time : 10 Feb 21 03:55:02
    Date and time : 10 Feb 21 08:48:36
    Date and time : 10 Feb 21 12:23:00
    Date and time : 10 Feb 21 13:11:32

    Which means Squish can be fixed, but JAM has the same problem (I tested JAM with hpt from github master without any patch).

    Maybe a problem/bug within SMAPI (JAM jammed into the Squish API)?

    ---
    * Origin: . (2:280/464.47)
  • From andrew clarke@3:633/267 to Oli on Sunday, February 14, 2021 23:44:06
    14 Feb 21 11:28, you wrote to me:

    I tested it with a JAM base and it still works. But it turned out a
    rescan from a JAM base also modifies time stamps:

    Original:
    Date and time : 09 Feb 21 06:31:33

    After rescan from JAM:
    Date and time : 09 Feb 21 06:31:32

    Which means Squish can be fixed, but JAM has the same problem (I tested JAM with hpt from github master without any patch).

    Maybe a problem/bug within SMAPI (JAM jammed into the Squish API)?

    api_jam.c:JamReadMsg() has this:

    scombo = (SCOMBO *)(&(msg->date_written));
    scombo = TmDate_to_DosDate(s_time, scombo);
    /* ftsdate = msg->__ftsc_date; */
    ftsdate = (unsigned char *)sc_time(scombo, (char *)(msg->__ftsc_date));

    But maybe the correct code should be:

    if (*msg->__ftsc_date)
    {
    ftsdate = msg->__ftsc_date;
    }
    else
    {
    scombo = (SCOMBO *)(&(msg->date_written));
    scombo = TmDate_to_DosDate(s_time, scombo);
    ftsdate = (unsigned char *)sc_time(scombo, (char *)(msg->__ftsc_date));
    }

    --- GoldED+/BSD 1.1.5-b20180707
    * Origin: Blizzard of Ozz, Melbourne, Victoria, Australia (3:633/267)
  • From Oli@2:280/464.47 to andrew clarke on Sunday, February 14, 2021 16:47:37
    andrew wrote (2021-02-14):

    14 Feb 21 11:28, you wrote to me:

    I tested it with a JAM base and it still works. But it turned out a
    rescan from a JAM base also modifies time stamps:

    Original:
    Date and time : 09 Feb 21 06:31:33

    After rescan from JAM:
    Date and time : 09 Feb 21 06:31:32

    Which means Squish can be fixed, but JAM has the same problem (I
    tested JAM with hpt from github master without any patch).

    Maybe a problem/bug within SMAPI (JAM jammed into the Squish API)?

    api_jam.c:JamReadMsg() has this:

    scombo = (SCOMBO *)(&(msg->date_written));
    scombo = TmDate_to_DosDate(s_time, scombo);
    /* ftsdate = msg->__ftsc_date; */
    ftsdate = (unsigned char *)sc_time(scombo, (char
    *)(msg->> __ftsc_date));

    But maybe the correct code should be:

    if (*msg->__ftsc_date)
    {
    ftsdate = msg->__ftsc_date;
    }
    else
    {
    scombo = (SCOMBO *)(&(msg->date_written));
    scombo = TmDate_to_DosDate(s_time, scombo);
    ftsdate = (unsigned char *)sc_time(scombo, (char
    *)(msg->__ftsc_date)); }

    Mhh, a JAM message base doesn't have an __ftsc_date field, I don't understand this piece of the code.

    What about the TmDate_to_DosDate conversion?

    union stamp_combo *_fast TmDate_to_DosDate(struct tm *tmdate, union stamp_combo
    *dosdate)
    {
    if(tmdate && dosdate)
    {
    dosdate->msg_st.date.da = tmdate->tm_mday;
    dosdate->msg_st.date.mo = tmdate->tm_mon + 1;
    dosdate->msg_st.date.yr = tmdate->tm_year - 80;

    dosdate->msg_st.time.hh = tmdate->tm_hour;
    dosdate->msg_st.time.mm = tmdate->tm_min;
    dosdate->msg_st.time.ss = tmdate->tm_sec >> 1;
    ^^^^^^^^^^^^
    }

    return dosdate;
    }

    AFAIK the JAM format doesn't use DOS Time at all. It seems to be an unnecessary
    lossy conversion.


    * Origin: . (2:280/464.47)
  • From Michael Dukelsky@2:5020/1042 to andrew clarke on Sunday, February 14, 2021 22:50:12
    Hello andrew,

    Sunday February 14 2021, andrew clarke wrote to Oli:

    api_jam.c:JamReadMsg() has this:

    scombo = (SCOMBO *)(&(msg->date_written));
    scombo = TmDate_to_DosDate(s_time, scombo);
    /* ftsdate = msg->__ftsc_date; */
    ftsdate = (unsigned char *)sc_time(scombo, (char *)(msg->__ftsc_date));

    But maybe the correct code should be:

    if (*msg->__ftsc_date)
    {
    ftsdate = msg->__ftsc_date;
    }
    else
    {
    scombo = (SCOMBO *)(&(msg->date_written));
    scombo = TmDate_to_DosDate(s_time, scombo);
    ftsdate = (unsigned char *)sc_time(scombo, (char *)(msg->__ftsc_date));
    }

    "ftsdate" variable is assigned but it is never used, so I've already commented it out but not in master branch yet.

    Michael

    ... node (at) f1042 (dot) ru
    --- GoldED+/LNX 1.1.5-b20180707
    * Origin: Moscow, Russia (2:5020/1042)
  • From andrew clarke@3:633/267 to Kai Richter on Tuesday, February 16, 2021 15:30:28
    15 Feb 21 21:39, you wrote to me:

    The bug is that the normal scan and the rescan created different
    DateTime.

    I don't understand what you are doing but it looks like you do some re-invention. Isn't it easier to look how the scan.c does the export and use the same code for the rescan?

    To summarise:

    "toss" shouldn't need the message base, instead just memcpy()ing the ftscdate field between .PKTs. This is how PassThru areas work but should also be true for non-PassThru.

    "scan" only exports local messages which in the case of a Squish base, only the
    DateWritten field is supposed to be written to by the mail reader, though Oli recently said that GoldED apparently writes the ftscdate field as well (which then means you can have a one second discrepancy between fields for the same locally written message...).

    "rescan" exports both local and non-local messages, which can have a combination of DateWritten field ONLY or DateWritten-and-ftscdate, and HPT incorrectly uses the DateWritten field for non-local messages, which only has a
    two second resolution in Squish.

    I can't say any more about JAM other than what I've already written.

    The *.MSG format stores the date solely in ftscdate format, so HPT should really have no choice but to memcpy() the ftscdate field into the outbound PKT on a rescan.

    So obviously the moral of the story is for everyone to use *.MSG. ;)

    --- GoldED+/BSD 1.1.5-b20180707
    * Origin: Blizzard of Ozz, Melbourne, Victoria, Australia (3:633/267)
  • From Kai Richter@2:240/77 to andrew clarke on Tuesday, February 16, 2021 16:29:02
    Hello andrew!

    16 Feb 21, andrew clarke wrote to Kai Richter:

    Isn't it easier to look how the scan.c does the export and use the
    same code for the rescan?

    To summarise:

    "toss" shouldn't need the message base, instead just memcpy()ing the ftscdate field between .PKTs. This is how PassThru areas work but
    should also be true for non-PassThru.

    Thanks, i hadn't that in my mind. I thought htp toss would toss into the messagebase only. To be honest i never paid attention at which run the downlink
    packages were build. Oh wait, are you talking about toss.c the code but not the
    function hpt toss?

    Regards

    Kai

    --- GoldED+/LNX 1.1.4.7
    * Origin: Monobox (2:240/77)
  • From andrew clarke@3:633/267 to Kai Richter on Wednesday, February 17, 2021 19:40:02
    On Tue 2021-02-16 16:29:02, Kai Richter (2:240/77) wrote to andrew clarke:

    Isn't it easier to look how the scan.c does the export and use the
    same code for the rescan?

    To summarise:

    "toss" shouldn't need the message base, instead just memcpy()ing
    the ftscdate field between .PKTs. This is how PassThru areas work
    but should also be true for non-PassThru.

    Thanks, i hadn't that in my mind. I thought htp toss would toss into the messagebase only. To be honest i never paid attention at which run the downlink packages were build. Oh wait, are you talking about toss.c the code but not the function hpt toss?

    Both I suppose. hpt toss is implemented by toss.c so they're one and the same.

    It's possible to have a completely passthru HPT configuration without a messagebase, with the exception of NetmailArea/DupeArea/BadArea.
    Though if you wanted to go full Marie Kondo you could point those to /dev/null!

    --- GoldED+/BSD 1.1.5-b20180707
    * Origin: Blizzard of Ozz, Melbourne, Victoria, Australia (3:633/267)