Botching a data import can ruin all subsequent data analysis. Raw data import processes are very sensitive and finicky based on the type of data that is being imported and the structure of that data. For instance, if you are importing a text file with city, states, and zip codes, you may run into an issue during your data review because most likely, Excel will have dropped the leading 0’s from the zip codes. Ugh.
Let’s say that data import is a step in a larger process rather than the end game and you’re building a sales forecasting model to analyze monthly raw sales data, and then make predictions about future sales goals. Every month, there is new data released, and therefore, everytime the end user opens the workbook and executed the macro, it would be a best practice to prompt them to import new data.
When you are importing data into excel from an external data source, you have a couple of options:
- Option 1: Opening the raw data file through the Excel File -> Open process which enables you to select how to each column is integrated and opens the file in a new workbook. This will walk you through steps defining for Excel how the data file is structured including the delimiters used, where to start the import, characters used for delimiting, and finally the column by column data format assignment
- This process is fairly consistent across Excel’s external data import tools, however the open functionality does not readily translate into a macro unless you were to have a separate file open that could then be referenced to copy the data values. This would be prone to errors and require a savvy end user
- Option 2: Use QueryTable to import the external data directly into the existing workbook
- The QueryTable process is predicated on the raw data not changing in format or structure. If your data is prone to changes in structure, you may need to go through a couple of additional steps to define some of the values that I will explain below
Given the robustness and customization options for QueryTable, this is my preferred choice.
I have broken my larger subroutine (ImportData) into two parts:
- Part 1: Import File Selection (the process of allowing the user to choose which file to import)
- Part 2: QueryTable set up for the selected file (the process of Excel processing the selected file and placing the formatted file into the existing workbook)
Part 1: Import File Selection
The below first half of the subroutine opens up a FileDialog box that enables the user to select one txt file, and then at the bottom of this excerpt of code, there are some helper functions that for instance: extract just the file name. This is helpful as we move to the QueryTable portion because setting variable such as the file path, and the file name then makes the QueryTable code more dynamic.
<pre>Sub ImportData()
Dim txtFileNameAndPath As String
Dim ImportingFileName As String
Dim SheetName As Worksheet
Dim fd As Office.FileDialog
Set fd = Application.FileDialog(msoFileDialogFilePicker)
With fd
'Enable this option if you want the use to be able to select multiple files
.AllowMultiSelect = False
'This sets the title of the dialog box.
.Title = "Please select the file."
'Sets the associated filters for types of files
.Filters.Clear
.Filters.Add "txt", "*.txt"
.Filters.Add "All Files", "*.*"
' Show the dialog box. If the .Show method returns True, the
' user picked at least one file. If the .Show method returns
' False, the user clicked Cancel.
If .Show = True Then
txtFileNameAndPath = .SelectedItems(1)
Else
MsgBox "Please start over. You must select a file to import"
'You don't want the sub continuing if there wasn't a file selected
Exit Sub
End If
End With
'Extracts only the file name for reference later
ImportingFileName = Right(txtFileNameAndPath, _
Len(txtFileNameAndPath) - InStrRev(txtFileNameAndPath, "\"))
When you are looking to implement this code in your own workbook, it would be good to make sure that the user has selected the correct file. For instance, you can search for words within the file name, you can verify the import afterwards, etc. The end goal is to ensure that the user doesn’t import data that can’t be processed. In my implementation, I also called a MsgBox immediately before calling this subroutine that explained to the user what was about to happen and the types of file the user should select to import.
Part 2: QueryTable Set Up
The second part of this subroutine takes the selected file, and then leverages QueryTable to consistently import the file.
Couple things to note about the below code:
- The worksheet where the data will ultimately reside needs to be the active sheet prior to QueryTable
- The Destination and the Active worksheet should be the same sheet
- Values such as TextFileTextQualifier, TextFileStartRow for instance, should be updated based on the way your data is structured
- TextFileColumnDataTypes will need an entry for every column that is being imported. Values should be used to differentiate between general, text, etc.
'You need to be on the active worksheet for the below code to work
Worksheets("Sheet 1").Activate
With ActiveSheet.QueryTables.Add(Connection:= _
"TEXT;" & txtFileNameAndPath _
, Destination:=Worksheets("Sheet 1").Range("$A$1"))
.Name = "ImportingFileName"
.FieldNames = False
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.TextFilePromptOnRefresh = False
.TextFilePlatform = 437
.TextFileStartRow = 2
.TextFileParseType = xlDelimited
.TextFileTextQualifier = xlTextQualifierDoubleQuote
.TextFileConsecutiveDelimiter = False
.TextFileTabDelimiter = False
.TextFileSemicolonDelimiter = False
.TextFileCommaDelimiter = True
.TextFileSpaceDelimiter = False
'Fill in the array in TextFileColumnDataTypes according to
'the type of data types that you are looking to import
.TextFileColumnDataTypes = Array(1,1,1,1,1,1,1,1)
.TextFileTrailingMinusNumbers = True
.Refresh BackgroundQuery:=False
End With
End Sub
I found the best way to get started with QueryTables, is to record a macro as you go through the data import process and then edit the macro accordingly. This will save you some time as values such as TextFileColumnDataTypes will be automatically populated.
If you have any feedback or additional ideas to make this a more seamless process, please feel free to share.
Dear Mr. Sherman,
I am looking for a way to open .csv files in excel so that it takes the data type as “text”. I found your site and think that you could maybe help me…
I was given a macro similar to yours (see below), with the command
.TextFileColumnDataTypes = Array(2,2,2,2)
-> works for a table with four columns.
I do not know the VBA language, but expected to be able to change Array(2,2,2,2) into a vector of lenght X (X being the number of columns of the imported csv table) and value 2.
So basically to steps:
(1) storing the number of columns of the imported chart into an object (X for example),
(2) creating a vector of length X and value “2” X times.
is there a simple way to do that in VBA?
thank you for your help!
Best regards,
Marion
The macro I have:
Sub CSVImport()
Dim fStr As String
fStr = Application.GetOpenFilename
If fStr = “False” Then
Exit Sub
End If
Dim truc As String
truc = UBound(fStr, 1)
Dim wkbReport As Workbook
Set wkbReport = Workbooks.Add
With ActiveSheet.QueryTables.Add(Connection:= _
“TEXT;” & fStr, Destination:= _
Range(“A1”))
.Name = “Data”
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.TextFilePromptOnRefresh = False
.TextFilePlatform = xlMacintosh
.TextFileStartRow = 1
.TextFileParseType = xlDelimited
.TextFileTextQualifier = xlTextQualifierDoubleQuote
.TextFileConsecutiveDelimiter = False
.TextFileTabDelimiter = False
.TextFileSemicolonDelimiter = True
.TextFileCommaDelimiter = False
.TextFileSpaceDelimiter = False
.TextFileColumnDataTypes = Array(2, 2, 2, 2, 2, 2)
.Refresh BackgroundQuery:=False
‘.UseListObject = False
End With
‘wkbReport.SaveAs (fStr & “.xls”)
End Sub
This is great macro. I am trying to adapt to my requirement but having some problem.
I have CSV file with 1920 columns. I want to import roughly 70 columns into excel. I tried to modify Array in .TextFileColumnDataType by following value but it exceeds the Module editor’s width.
Is there an alternative to resolve this issue?
2,2,2,2,2,9,9,2,2,2,9,9,9,2,2,2,2,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,2,2,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9
thanks
-gobar
You can change the separator character that is used in both delimited and .csv text files. This may be necessary to make sure that the import or export operation works the way that you want it to.
I am trying to import a text file to a specific worksheet.
Sheet 1 is to be a cover sheet with manipulated data based on several input fields from a drop down list.
Sheet 2 is the actual calculations manipulating the data. I can input the data but cannot get it to input to Sheet 2 if I am on Sheet 1. Is this possible?
Hi Mr Sherman, just wanted to thank you for making your code available on the net, it is very helpful to me in a project I was a bit stuck on and gave me a great solution. really appreciate it mate!
Kindest regards,
Pete
Thank you for this helpful article! It saved a lot of time for me!
This import works but how do I stop text wrapping on the imported data as it is causing problems down the line in other code..?
Sub CSV_Import()
Dim ws As Worksheet, strFile As String
On Error GoTo ErrorHandler
Application.ScreenUpdating = False
Sheets(“Import”).Visible = True
Set ws = ActiveWorkbook.Sheets(“Import”) ‘destination sheet
strFile = Application.GetOpenFilename(“Text Files (*.csv),*.csv”, , “Please select text file…”)
‘pulls in all fields so dump this on Import sheet and copy and paste from there to WalkData
With ws.QueryTables.Add(Connection:=”TEXT;” & strFile, Destination:=ws.Range(“A1”))
.TextFileParseType = xlDelimited
.TextFileCommaDelimiter = True
.Refresh
End With