We have 10 servers. Some flight related data will be returned to the server. Data from the server will come to our application. The same data may arrive on our application for more than one time, but in the end I need to save that data only once in the database. So we are checking the database before inserting the data. If that record is not already present in the database, then only we are going to save the data. But for some reason we are getting duplicate records in the database.
Is it necessary to use synchronization in this scenario.
Thank you in advance what may be the problem here ...
Our company In the way we deal with more than one data source where only one piece can go through information using batches. What we got is what we are doing on the code level (Java and .NET), we want to invest a lot of time and still have to repeat.
By implementing the batching process, we store everything locally and 2 batch jobs 1 will ensure the quality of the data and remove the duplicates 2 will compress the compression service and the second data (we Use the XCOM to push directly into the DB queue, which then plug the data again).
If you can apply something like this, you have a central point of entry on which you can apply the proper quality gates.
Hope our example helps, if not let me be happy to remove it. :)
Comments
Post a Comment