Sage 100

 View Only
Expand all | Collapse all

Merge / Delete Items

  • 1.  Merge / Delete Items

    Posted 01-19-2023 19:18
    I am dealing with a large database and have encountered extreme slow performance when attempting to use the Delete and Change utility.  By my calculations it would take 2 1/2 years to complete - that's with purging item transaction history and only keeping the last 3 years!

    Has anyone achieved this using a SQL script?

    ------------------------------
    Andrew Hagenbach
    Hagenbach Solutions, LLC
    ------------------------------


  • 2.  RE: Merge / Delete Items

    Posted 01-28-2023 14:37
    @Andrew Hagenbach  Did you find an alternate solution for this?  I am doing one this weekend.  It took 1 hour to complete the first 20.  I have been asked to remove any item with a last receipt date prior to 1/1/2021.  There are 140,000 items that meet this criteria.   So, yes, we could be here a while.  3 minutes per item, so 420,000 minutes, or 7000 hours, or 292 days.... and this is premium.  Should I ask the owner if they can hold of on invoicing and receipt of goods for 292 days until the merge completes?


    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 3.  RE: Merge / Delete Items

    Posted 01-29-2023 20:27
    If an MD can post a list of the tables that the utility hits for deletes, and what the "beginning balance" record is that gets created, it should be possible to at least do this for deletes.  Do they want history purged for the items you are keeping?

    Phil

    ------------------------------
    Phil McIntosh
    President
    Friendly Systems, Inc.
    ------------------------------



  • 4.  RE: Merge / Delete Items

    Posted 01-30-2023 01:49
    @Phil McIntosh  You can't do a delete if there is any history for an item...  which is all of them.  You have to do a merge to another item such as ZZZDELETE, which effectively deletes the item.   During the processing there are table name descriptions that display. For example, PO_PurchasesHistory displays.  This isn;t a table name so I imagine it means PO_PurchaseHistoryDetail.  Also displayed is PO_ReceiptHistoryLotSerial, SO_SalesOrderDropShip, AR_InvoiceHistoryDetail, RA_ReturnDetail, SO_PackageTracking, and RA_DeleteRenumberMergeXref.  There is probabl;y more.  Can an MD confirm ?

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 5.  RE: Merge / Delete Items

    Posted 01-30-2023 01:53
    @Phil McIntosh  If you could use your idea to do an actual delete then that would be great.  Why should we care about beginning balances if the item is deleted?  Beginning balance of what? ​

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 6.  RE: Merge / Delete Items

    Posted 01-30-2023 07:13
    @Doug Higgs - that was why my question of delete only.  That only needs list of tables to delete records from.  The history purge is trickier.​

    ------------------------------
    Phil McIntosh
    President
    Friendly Systems, Inc.
    ------------------------------



  • 7.  RE: Merge / Delete Items

    Posted 01-30-2023 08:44
    @Phil McIntosh  I think the "history purge" is just a find and replace, or in SQL terms a select and update.  I don;t understand why it's taking so long if the db is SQL.  Is Sage using the business objects to select and update instead of calling a stored procedure that uses TSQL?  Is that the reason.  Seems to me a simple statement like below on each table would do it.
    UPDATE 
        YourTable
    SET 
        Column1 = REPLACE(Column1,'a','b')
    WHERE 
        Column1 LIKE '%a%'
    



    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 8.  RE: Merge / Delete Items

    Posted 01-30-2023 10:11
    Sage uses the same code across all editions, so the SQL ends up being RBAR (Row by agonizing row)

    ------------------------------
    Phil McIntosh
    President
    Friendly Systems, Inc.
    ------------------------------



  • 9.  RE: Merge / Delete Items

    Posted 01-30-2023 10:45
    That figures.  You would think PVX could call a stored procedure that executes 100 times faster.  This would be seemingly just a few lines of T-SQL.

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 10.  RE: Merge / Delete Items

    Posted 01-30-2023 11:05
    SQL could be faster for sure, but it isn't as simple as a straight UPDATE command.  A number of tables would require special handling to merge rows (like IM_ItemWarehouse, IM_ItemCustomerHistoryByPeriod...) or you can run into primary key violations.  Oh, and PO_PurchasesHistory is a real table.

    To get a full list of tables, below is a key word search for column names (where you have to know the system and manually check for alternative names like BillNo...).
    SELECT      c.name  AS 'ColumnName'
                ,t.name AS 'TableName'
    FROM        sys.columns c
    JOIN        sys.tables  t   ON c.object_id = t.object_id
    WHERE       c.name LIKE '%ItemCode%'
    ORDER BY    TableName
                ,ColumnName;
    ​


    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    ------------------------------



  • 11.  RE: Merge / Delete Items

    Posted 01-30-2023 13:05
    Thanks @Kevin Moyes.  Yes, it seems to be more complex than a simple select and update.  It may be better to talk the customer into deleting a lot of history to speed up the merge process.


    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 12.  RE: Merge / Delete Items

    Posted 01-30-2023 13:14
    We've had clients inquire about a big cleanup of item lists, but then abandon the effort when the insane processing time becomes clear.  (I think one estimate came out to being years, not "just" months).
    Changing items to Inactive, and hiding them from ALE's seems to be Sage's way to deal with this, instead of the massive data crunching to properly do the cleanup.


    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    ------------------------------



  • 13.  RE: Merge / Delete Items

    Posted 01-30-2023 21:53
    @Kevin Moyes  re your comment " A number of tables would require special handling to merge rows (like IM_ItemWarehouse, IM_ItemCustomerHistoryByPeriod...) or you can run into primary key violations".  All of the items that are being merged have 0 qty on hand.  Is there any reason that IM_ItemWarehouse can't be skipped.  No need to merge 0 qty records. Same with IM_ItemCustomerHistoryByPeriod... Why bother?  It may make sense to change the item code in AR_InvoiceHistoryDetail and PO_PurchaseOrderHistoryDetail but there would not be a merge needed because the item code isnlt part of the key.

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 14.  RE: Merge / Delete Items

    Posted 01-30-2023 22:01
    @Kevin Moyes  Not completely skipped because the record would still need to be deleted.  Existing records wouldn't need to be changed.​

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 15.  RE: Merge / Delete Items

    Posted 01-31-2023 10:34
    Those were just two examples.  You'd have to come up with a plan for every table like that, according to what you need (with an eye towards GAAP...).

    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    ------------------------------



  • 16.  RE: Merge / Delete Items

    Posted 01-31-2023 10:37
    What's GAAP?  Rings a bell.

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 17.  RE: Merge / Delete Items

    Posted 01-31-2023 10:41
    Generally Acceptable Accounting Principles.
    Using Sage utilities I can just point at Sage, but for every manual data edit, I think of what auditors might say.

    ------------------------------
    Kevin Moyes
    Technical Systems Analyst
    Munjal White Consulting Co.
    ------------------------------



  • 18.  RE: Merge / Delete Items

    Posted 01-31-2023 10:57
    You would definitely want to make a copy of the company before doing this.

    ------------------------------
    Phil McIntosh
    President
    Friendly Systems, Inc.
    ------------------------------



  • 19.  RE: Merge / Delete Items

    Posted 01-31-2023 11:09
    @Phil McIntosh  Yea.... I'm not doing this.  Customer has agreed to let me import a Y to the inactive field and leave it at that for now.  Just an interesting conversation and another area where Sage could improve the premium product with a little forethought on how to take advantage of SQL.​

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 20.  RE: Merge / Delete Items

    Posted 01-31-2023 11:04
    Principles sminsiples.  Rules rules rules.

    ------------------------------
    Doug Higgs
    Midwest Commerce Solutions, Inc
    (312) 315-0960
    Chauffeur, Chef, and Personal Assistant to Sprinkles
    ------------------------------



  • 21.  RE: Merge / Delete Items

    Posted 01-30-2023 11:13
    All,

    Good topic to add to Sage Product Improvement requests.
    Their Sage 100 2023 and 2024 roadmaps include:
    "Selected usability improvements and Customer requests"
    They are looking for small improvements to jazz up the versions.

    Lee Graham
    --

    Friendly Systems, Inc.

    Visit us on the web at www.friendlysystems.com
    Office:?? 678.273.4010 ext 4
    Mobile: 404.202.9065
    Fax:?????????? ??678.273.4012
    See our blog at www.friendlysystems.com/blog


    Virus-free.www.avg.com





  • 22.  RE: Merge / Delete Items

    Posted 01-30-2023 15:08
    That wouldn't be small - it would require Sage to actually learn SQL...

    ------------------------------
    Phil McIntosh
    President
    Friendly Systems, Inc.
    ------------------------------