August 31 2007 • 8:12 PM
By:
Michael Murphy
The InDesigner – Episode 43: Data Merge
Who says data and design are mutually exclusive? Not InDesign, that’s for sure. Under the right circumstances, data-driven layout is the way to go. Unlike XML, which makes designers run from their computers, the Data Merge feature in InDesign is an easy way to accomplish fast, flexible data-driven layout creation and updates. Using a subscriber’s files as an example, this episode takes a video look a topic that was tackled way back in the audio-only days of Episode #7.
Download it now or watch it in your browser:
The InDesigner – Episode 43 (15:00 mins. | 28.6 MB)
Remember you can also data merge Snippets just like images, especially if you need different coloured background panels in your design.
That’s news to me, but it makes perfect sense. And it’s a fantastic way to push the feature in a creative way. Thanks!
Thanks Michael! I meant to ask before – can you merge data from multiple CSV files into a single InDesign document or not?
So now I want to know how to use snippets that way!
Excellent videocast Michael!
Thanks alot,
Wonderful cast, but I just want to correct something if you dont mind, the correct link to Steve’s Data Merge Posts,
http://indesignsecrets.com/automating-data-handling.php
Thanks again
Anybody know how to export/save Unicode text (in this case Cyrillic text) from MS Excel or Numbers to use in a ID CS3 data merge?
My current solution is to open XP notepad through Parallels copy/paste and save as Unicode. There has got to be a better way.
Looking forward to your suggestions!
I want to know, if the excel file can link
A)AI file?
B)If I have 3 items on this page, and 5 items on other page, will the dataMerge able to handle?!
C)Can I put Non -English in the data?!
Thanks a lot for your help, it is so useful!
Sorry! missing one point, if I have my layout with left and right page designed, can I still DataMerge.
To use snippets or illustrator files, just add the ‘@’ character in the header row like you would for an image.
Hi, for a demo I’d create similar file. The problem was that when the files are changed of computer, the path of the images is not good any more. I found a way (on macintosh) of circumventing this problem. With Disk Utility, you can create an empty DMG (with wright/read autorisation) and copy in all your merging files (pictures, INND, TXT…). Now update all path links in the Excel Files and it’s always OK.
Best
Like Andrew, I have a document with left and right hand pages. One record per page, but the left hand page is white with black text and the right hand page is black with white text, one record per page. Is there a way to make this work using data merge? Or do you need two make two indesign documents?
Great podcast again! But, just to be difficult, here’s a question that’s slightly off-subject in a related kind of way: rather than merging, can you link to external data (such as txt files), design away and then have InDesign (I only have CS2, sadly) update just the data when you make changes to the external file. When I have been playing with this kind of thing, all I can see is is the function that allows you to place the data as a linked file, but over-writes your post-placement formatting each time you update the link. Any suggestions?
Hi, everyone. I’ve been away this weekend and am just catching up on your many questions. Here goes:
1) Colin — Each merge needs to be a separate operation. There can only be one data source in the Data Merge palette for any given document.
2) Taysh — low Jackson actually replied to this (thanks!). Anything that references a file needs to have the “@” symbol at the start of the header row name, just as you do with image references.
3) Mamoon — thanks for including the correct link. I will try to get a “patch” edited into the episode that corrects this so that people who download it in the future will have the correct info.
4) Thatcher — This is a puzzler for me…it’s not something I’m familiar with, but I will ask around
5) Andrew — (a) Illustrator files can be referenced in a column with “@” at the start of the header row name, just as I demonstrated with PSD files; (b) Data Merge is all about consistency…that’s a limitation you just have to deal with. If you have three items on one page and five on another, the only way to achieve this is to set your template up as five per page, then add some “dummy” records to your Excel file, do the data merge, then delete the empty records in the resulting layout. And, unfortunately, setting up spreads with inconsistent layouts will not work with Data Merge. As you saw in the episode, only one placeholder layout can be created, and it’s margins and space between settings that determine how many fit on a page; (c) as far as InDesign is concerned, text is text, so you should be able to use any language in your data. There may be some issue with non-Roman characters, but I don’t know enough about that to say for sure.
6) Liz — As I said to Andrew, Data Merge is limited to a rigid consistency. If you have alternating layouts, you can create two merge files, then combine them after the fact. Alternately, if your layout is consistent except for the color of the text and background, consider using Object Styles that allow you to quickly re-format those entries in the final layout. Neither is a one-step option, but they’re the best recommendations I can make.
7) Robin — Unfortunately, the problem with linked text losing its formatting still exists, even in CS3. There have been improvements with tables linked to Excel files if you use table and cell styles, but that’s table-specific. Linked text files will not maintain their formatting when updated.
Could you please add images and stuff you use for downloading include the video? Thanks!
Great tutorial video! I am disappointed to see that multiple records on a single page are not actually “mergeable,” but are just a new document. Alas!
Thanks for this great tutorial! I don’t personally have Excel on my Mac, where I have InDesign. I got around this so I could do a data merge test by using Google’s “Documents” feature, where you can create your own spreadsheet as long as you have a Google account.
I’ve done data merge from Excel to Word on the PC, so this was a very interesting experiment.
I’m still trying to figure out if it’s possible to flow data into linked text boxes, instead of having a text box for each recordset.
Wow, what a mind-blower! I’m new to scripting & InDesign over all, but can’t stop finding new & exciting things to learned. Thanks Micheal. Can’t wait to see the next episode.
I am looking at a 10000 record data merge, ten thousand names and addresses for a postcard job.
Whatcha think Mike? 2500 pages of a 4-up postcard of indeterminate art? Should I split the data up in groups? 500 page runs? Other than having a lot of RAM, What advice do you have before I jump off a cliff.
thanks!
Bob
While I recognize that Data Merge is a useful tool for basic mail-merge jobs, users should aware of it’s limitations — it’s certainly not a replacement for third-party InDesign VDP plug-ins (Darwin, uDirect, InData and others). I’ve written a post which explains the pros and cons of personalization features in Creative Suite (InDesign, Photoshop, Illustrator and Dreamweaver) which may be of interest to readers. Check out: http://www.veedeepee.com/2007/10/free_vdp_1.html
Bob — Sorry it’s taken me a month to answer your question. You’ve probably already printed and mailed the postcard job already but your approach is perfectly sound. I’m sure it went off well for you 4-up is a great way to keep the number of pages to a sane 2500, and splitting the data up may just make things easier for you to troubleshoot if something goes wrong. You can always use a subset of your data to work out all the merge kinds, then use the full data set as your data source when you know it’s good to go. Best of luck.
holy guacamole
It looks like you might have just saved my weekend here. Really clear and easy to understand instructions which I’m hoping will make an immensely tedious job FLY by. We shall see…
Boy, do I love to hear that! Thanks, Ed. Good luck on your project.
Great piece on data merge! However, I have a new user question that I cannot seem to answer myself. In your example, you have a blue bar graphic in the footer area of the master and resulting merged document. I have been trying to reproduce a similar design element in my documents and it seems to “break” the merge in that it only puts one record per page instead of the 30 or so records per page (2 columns) that I intend to have at the end. What am I missing?
Kristi — Are the elements you intend to merge also on the master page? If that’s the case, you’ll get the result you describe: one recored per page and a new page generated for each record. If you want a multiple-record layout, your placeholders, and the frames that contain them, must be on a document page, and you need to set it up so that it will cleanly and evenly fit the number of records you want when InDesign starts making duplicates during the merge process.
I have a document with the place holders for the variables to be merged in to a resulting document (multiple pages). If I use preview with the multiple record layout, it shows it perfectly. However, if I go back to the main document and add anything (e.g. current page number marker) to the margins (in header/footer areas), it no longer repeats the records in the preview, but shows them one per page.
Kristi — I think I’d need to see the file(s) themselves to really troubleshoot this. Feel free to e-mail them to info@theindesigner.com
you DID save my weekend… and made the week after a great deal easier too. Thanks again for the great tutorial!
Did you get my files? Any idea what’s up (or what stupid thing I did /didn’t do)?
Data Merge just broke the record for me biggest time saver applied to date. THANK YOU SO MUCH for introducing this to me.
I just want to say thank you for this video! You have literally saved my butt! I could not for the life of me remember how the multiple records function worked and your video layed it all out in a clear and concise manner. Thanks again!
I love this tutorial! I’m currently working on my college yearbook and I’ve finally figured out how to use data merge, but I was wondering if there was a way I could make the photos go horizontal while the text lines up in columns (like a typical yearbook class photo page). Thanks!
Rachael — Data Merge’s Multiple Record Layout options allow you to choose whether your additional records get put in left-to-right or top-to-bottom. It’s your choice. When setting up the merge, in the Create Merged Document dialog, under the Multiple Record Layout tab, choose Arrange by Rows First in the “Layout of Records” area.
Hi, Mr. Murphy,
Have you ever encountered the problem of having an error when you select the data source?
The error mesg says “The selected data source has one or more empty field names. Please fix the file or select another file.”
In my exel file:
1) A~G (7columns) x 109 Rows have no blanks and filled with data
2) There are no line breaks in each cell
3) contains symbols: # &
4) contains space in each cell
5) contains no formula/function
Also, I’ve tried both txt and csv. There’s only one layer in InDesign.
My co-worker was able to merge the data few hours ago, but now the error mesg is back again. I’ve used data merged before and never had this problem.
This might still be a very vague description for my problem, but thank you very much for your help.
Hello,
I found my problem. I opened the csv file in textEdit and there are multiple commas at the places that supposed to be just one comma
example,example,example,,,,
example,example,example,,,,
example,example,example,,,,
so I deleted them
example,example,example,
example,example,example,
example,example,example,
and now the data can merge.
I don’t really know what created that though…
Thanks anyway.
Hi Michael,
I have set up my table, inserted the fields for the merge (on the actual page, not a master page).
On records to merge I have selected ALL however the Multiple Record Layout Page is grayed with a message above indicates that options are disabled because I have selected single record!! I have repeatedly gone back into the Records page and everything is in order however Records per Document Page displays greyed in as Single Record! Yiikes, WHAT AM I DOING WRONG??!!
Trish — It could be any of a number of things. To narrow it down, send me a PDF of your layout (to info@theindesigner.com), so I can look for obvious causes first.
Michael,
I pounded through it yesterday and figured out that I had multiple pages prior to the merge. Once I fixed that the merge went somewhat ok. Data merged and 14 pages were generated however when I print using booklet feature, page position upper left (in setup) an extra blank page appears in print and the data is flipped as though it is to be top bound. I will send you the pdf of that file. I have looked at the data and can not figure out why it is creating an extra page. The blank page does not appear in the pages menu. Document setup Letter-half page. I must have something in conflict here! Thanks for your quick reply.
Hi Michael,
Well, I have to disclose the error of my ways and “steps” used to resolve creating this directory.
After reading the limitations on multiple records merge the *light* came on and I deleted the additional pages I had placed in the document and created a layout that would fit one record. The multiple records merge then worked.
IMO, this plays havoc in creating a saddle stitched directory with a cover, contents and ancillary material in the back. I believe I can add those pages after the merge but this is somewhat cumbersome for future updates each year if I am unable to leave that static material in place and simply merge the changed file to more easily automate this directory production.
Steps I used for 8.5″ x 11″ directory, duplexed.
1. Document Setup: Letter Half, Orientation Portrait
2. Set up Text Boxes or Table for fields into which data will be merged as it would appear for one record. (When the merge is accomplished, this format is applied to all multiple records coming in.)
3. Window Tab: Automation, Data Merge. Drag and drop or click and select data specific to the text or table designed.
4. Data Merge Menu (upper right corner of data merge screen) select drop down to display merge function.
5. Fix any errors due to undersized fields. Increase text box or target table size.
6. File, Print Booklet, 2-up Saddle Stitch, page position upper left, printer: Adobe PDF
7. To properly orient on the page when printing: select duplex, change page icon to landscape.
I always seem to have personal difficulty finding some of the tools, functions and menus that are discussed in the help file. Geees, just give us a clue! The Select, Update Remove Data Source and Create Merged Document screen is an example! It was so buried! Thanx, I hope this will help someone else trying to create a simple membership document without angst.
- Trish
Hello Michael;
I am encountering an error when i try and data merge an image. From a previous post I found that when I entered an @ before the file name, then removed it, it allowed me to merge the images! Then I noticed that I was truly missing images, I printed the report, found and moved the images into the correct folder, then tried to run data merge again. Now it will not find ANY of my images not matter what I do.
Can you assist?
Thank you
Allene
Allene — Based on what you wrote, it sounds like the problem is that you’re putting the @ symbol before the filename. That’s not where it goes. The @ symbol should be the first character in the header cell of any column containing image references. The image filenames themselves should be preceded by their path on your hard drive or network.
Michael;
I’ve another question. Regarding variants. We have threads that have a main part #, then each of these part #’s will have variant or color #’s. I need to put these variants into the indesign document as well, and want to use datamerge to do this. Is this possible? If so, can you assist? For an example, look at our website http://www.quiltersresource.com, then in the search field type in 816-05- you will see a listing of Presencia perle cotton… if you click on this to drill down you will get a list of all the color #’s these are the variants I am talking about. I need to get the regular data, part #, title, description, cost, retail, then list out these colors. Any clue as to how to do this?
Thank you
I have maybe a unique problem with my Data Merge that may not have a solution?
I produce a 4-page newsletter that printed on both sides, folded and stapled and each folded page size is 5.5″ x 8.5″. (A 16-page mini-magazine) I want to print the 950-name mailing list on the outer cover but have discovered that I have the option of printing a 7600 page document or manually printing each issue per address. Either way the printer (Firey Software) would have to rip those pages instead of ripping the issue once and repeating it 950 times. Is there another way to do this or am I stuff with this problem?
(The solution has been to just print labels by themselves and stick them on by hand)
Thanks you!
Ralph, this is a great example of how DataMerge isn’t for everyone! I would suggest you look at other third-party database publishing and variable data printing solutions.
Anyone know how and if it’s possible to…
1) Dynamically import another Indesign document into a placeholder (i.e. dynamically merge them together)
2) Use multiple master templates and make the .CSV file select which one to use?
Michael:
Another question!!! I need to have the images I am putting in a data merged document to fit to frame, figured that our, but to then anchor the picture at the bottom. So I want them aligned at the bottom of the frame, not just cropped and leaving white space bewteen the image and the bottom on where the image should be. Does this make sense?
Thanks
Allene
Leigh, Data Merge cannot change master pages, but I believe this is something that InData from emsoftware.com can do.
Allene, did you watch TheInDesigner #45? I think he included Frame Fitting Options. You can adjust the Alignment to the bottom, so that the image appears at the bottom of the frame.
Hi, I have tried datamerge to work with snippets on InDesign CS4 – Mac, but ID keeps telling me, that he cannot support this filetype when I want to see the preview. I have checked all the paths, but everything seems correct. Anyone got an idea what I am misssing ? Stefan
Hi Michael:
Just watched your video on data merge and it hit the nail on the head. I was stuck in the middle of a business card design (trying to merge the data from an excel file) and after watching, I completed the project in 15 minutes!! (Where do I send the check??)
Anyway, I have an ongoing need to create things like business cards and other variably printed items. These items need to be placed on a website so our customers can order them from us.
We’ve been using a third party portal and place whatever items our customer needs on the site, but the problem is that we have to go thru this company to set up variably printed items. They use PageFlex to set things up and it’s costly and takes too much time.
Is there a way to use our InDesign files and put them up on a website for our customer’s access? Ideally, the process would be for the customer to input the order information and the file would be proofed to them, then it’s transmitted to us or our plant with the job production information (JDF).
Any ideas on how to proceed?
Thanks
Barry
Barry –
What you’re describing sounds like a job for InDesign Server.
http://www.adobe.com/products/indesignserver/
Thanks for the reply Michael. I’ll check out the server product.
One other question for you…
What program did you use to create the video to show the data merge process?
I need to create a training video for our customers and put it on our website.
I really liked your format.
Thanks again,
Barry
Project back on the front burner! I am merging my data file into the InDesign template (business cards) and have hit a snag.
The card has 4 fields in the lower left corner:
Phone:
Fax:
Cell:
Email:
The values come from individual columns in my excel file. However, if there is no date (ie: no cell number) then that field is empty but the tag remains.
Any idea how to remove the tag if there is no associated data?
Thanks!
Thanks you so much … I’m doing wedding invitations and this is an excellent tutorial …
Thanks!
[...] InDesignSecrets ? Blog Archive ? The InDesigner – Episode 43: Data Merge __________________ Business Card Printing > Business Cards | Letterhead Printing > [...]
I would like to designate multiple PDF files as images, but Data Merge doesn’t seem to see it that way. The “Picture” box remains empty on preview. If I save the PDF’s as EPS files, it works perfectly. Any workaround to avoid having to convert hundreds of PDF’s?
…and thanks for the GREAT tutorial!
A significant issue with InDesign CS4’s Data Merge has been fixed in the 6.0.4 update. See the release notes for item #2335625.
Is there a way to link an InDesign data merge document to a Google doc without exporting it as CSV first? I’d like to link directly to the live Google doc.
Love your expertise!
-mike
No good news there, Mike. The only formats accepted for an InDesign Data Merge are CSV or TXT.
K! Thanks anyway. I’ll keep experimenting and let you know if I find a solution.
Since reading all of these comments I have found the answer to my problem!!! Thanks for the help!