Sage 100

 View Only
Expand all | Collapse all

Multi-Currency Conversion Premium to Premium

  • 1.  Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 10:45
    What is the longest anyone has seen a DSD multi-currency data conversion take? 

    I have a client finally migrating from 2013 to 2020 (both premium). They utilize DSD's multi-currency and I'm running the DSD Enhancement Control Panel > Convert Data for GLMC. The conversion has been grinding on GL_DetailPosting for 18 hours now. To be fair, that file is the largest I have ever encountered with almost 30 million (MILLION!) records. Task manager shows the SQL server is using 97% of the CPU. All of the other companies (much smaller size) converted successfully so I think the program is okay. I'm just curious to hear if anyone has run a conversion similar to this one and can tell me how long this might actually take. Thanks!

    ------------------------------
    Kate Krueger
    Business Applications Manager
    Eide Bailly, Inc.
    Denver CO
    970-692-5113
    ------------------------------


  • 2.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 11:15
    I've seen some long ones, 6+hrs but with nowhere near 30 million rows.  Premium seems to take longer for the DSD conversions for some reason too.  I just let it go until it's done, as long as the pvxcom.exe is still showing activity.
    Changing your SQL backup model to Simple during the conversion can speed things up... of course you can change things back to Full afterwards if necessary.

    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    Toronto ON
    ------------------------------



  • 3.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 11:23
    Hi Kevin! Here's the thing - I don't see any activity on the pvxwin32.exe that are in the task manager but sqlserver.exe is maxed out. Does that concern you?
    Is it too late to change the SQL backup model to simple since it is already running?

    ------------------------------
    Kate Krueger
    Business Applications Manager
    Eide Bailly, Inc.
    Denver CO
    970-692-5113
    ------------------------------



  • 4.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 11:25
    You can change the backup model anytime.  SQL handles it seamlessly.
    I think it's pvxcom.exe that does the conversions, not pvxwin32... but I might be thinking of the Sage migration / conversion instead of DSD conversion.

    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    Toronto ON
    ------------------------------



  • 5.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 11:32
    To elaborate of @Kevin Moyes comment, the backup model determines what gets written to the transaction log.   If the backup model is set to full then every transaction is writing to the transaction log.  With 30M records the server better have a lot of available hard disk space because the transaction log will most like be huge.


    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Assistant to the Traveling Secretary
    ------------------------------



  • 6.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 11:50
    Exactly.  Such a high SQL CPU number made me think of the logs, which is something the SQL service would have to handle.

    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    Toronto ON
    ------------------------------



  • 7.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 11:53
    I've changed the backup method. @Doug Higgs you are totally right. Early on the conversion complained because it filled up the hard drive. I cleared up some space ​but I haven't seen the free space reduce since I did that. If I look at the time stamp on the .LDF file it was altered about 10 minutes ago.  so I think it is still running but sheesh, no update for 10 minutes? I wish I felt better that it was running correctly... and I think the auto backup kicked on which totally messed with my time stamps. sigh.

    ------------------------------
    Kate Krueger
    Business Applications Manager
    Eide Bailly, Inc.
    Denver CO
    970-692-5113
    ------------------------------



  • 8.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 12:31
    Whenever I change from Full to Simple I always check the File Shrink screen for the Log, to verify the Available free space is high.


    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    Toronto ON
    ------------------------------



  • 9.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 12:46
    I'm showing 99% available free space - should I still shrink the log?

    ------------------------------
    Kate Krueger
    Business Applications Manager
    Eide Bailly, Inc.
    Denver CO
    970-692-5113
    ------------------------------



  • 10.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 13:02
    If it is insanely oversized or if you are low on disk space, sure.  The important thing for performance sake is that it isn't maintaining the logs while the conversion is processing.

    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    Toronto ON
    ------------------------------



  • 11.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 13:25
    I just checked the time stamps for the .mdf file in the Data folder and the time stamp is almost two hours ago even though sqlserver.exe has been pegging the whole time and the Sage UI just has a progress bar stuck at 100% (which is what it has showed for the last 15 hours). Does this indicate that it isn't actually doing anything? I know I'm being nuts about this one - this is 30+ hours of work over a weekend in the balance here.

    ------------------------------
    Kate Krueger
    Business Applications Manager
    Eide Bailly, Inc.
    Denver CO
    970-692-5113
    ------------------------------



  • 12.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-25-2021 13:29
    SQL is bypassing a lot of the Windows file property stuff when it writes the data (for the sake of efficiency).  The date / time on the MDF not being updated minute to minute is nothing to worry about.  From a random google:


    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    Toronto ON
    ------------------------------



  • 13.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-26-2021 04:07
    I'm very late to the conversation but one way of checking is do a SQL Profiler Trace (somewhere on the SQL Mgmt Studio menu). You can choose a default template for now and just turn on the trace and have it run for say 30 seconds before you click on Stop or Pause on toolbar. Among others, pay attention to the Duration column (shown in milliseconds I think) to see which SELECTs, INSERTs, or UPDATEs are taking a long time.  You might also be able to gather that it could be running in an endless loop (hope not), reprocessing the same rows repetitively, due to some condition in the data.

    Also it's great you changed the DB Recovery model to Simple to reduce the t-log impact and size but also check to see if Transaction Log Shipping might be enabled or SQL replication is running (pretty unlikely). Also check for (dare I say it) the infernal Change Tracking feature being enabled, the one you know well which can be set not only at the DB level but table level too.

    ------------------------------
    Alnoor Cassim

    Email: alnoor@asifocus.com
    Ph: 949-689-9887
    Orange County, CA
    ------------------------------



  • 14.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-27-2021 18:38
    Edited by Kate Krueger 01-27-2021 18:39
    Definitely not too late to this party as this conversion is apparently going to take forever to complete. I was able to use the SQL Server Profiler to see that it is running through the data and each record is taking a few milliseconds. Between the trace yesterday morning and another one this morning, it completed 5 GL accounts in GL_DetailPosting. If each record takes a second, 30 million records is going to take...um...almost a year. Hopefully it is faster than that! The client obviously had to go back to work in the old version so my next plan is to try and move some of this history to a history company and make the file smaller to convert.

    But @Alnoor Cassim i used your most excellent sql query to update the infernal track changes before I started this battle and it found around 15 tables where tracking was turned on. Thanks so much for putting the query out there and saving me the pain of having to find all those!

    ------------------------------
    Kate Krueger
    Business Applications Manager
    Eide Bailly, Inc.
    Denver CO
    970-692-5113
    ------------------------------



  • 15.  RE: Multi-Currency Conversion Premium to Premium

    Posted 01-27-2021 18:53
    You might be able to move the line data elsewhere, convert, then move it back... I had to insert data into GL_DetailPosting manually for a project once, and you should be OK as long as the SequenceNo field is unique (and handle any true data changes like initializing new fields).


    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    Toronto ON
    ------------------------------