This Command line tool only makes sense if you apply some more general best practices in flat file processing.

If you take 100 different files in 100 different formats from 100 different providers, there will be some specific issues with each file (such as downloading it from a location, handling a proxy, decompressing it, normalizing) and some generic issues (basically translating a file to a database table, converting strings to specific data types)

The idea is to handle the specific stuff in pre-processing scripts, and what can be made generic in BusyBulkCopy.

the interface between BusyBuklCopy and the pre-processing script is a rfc-4180 compliant csv file http://tools.ietf.org/html/rfc4180

BusyBulkCopy will:
  • match columns found in the file and columns found in the database table
  • Convert the strings found in the csv file to the destination table's data types
  • try to have a positive attitude towards errors that happen (truncate strings, round numerics, replace invalid values with null, etc.
  • handle issues such as quoted separators, quotes in quotes, line feeds in fields, etc.

Last edited Dec 5, 2014 at 3:08 PM by busytools, version 1