< back to full list of articles
Safe Passage Through the Data Supply Chain

or Article Tags




Safe Passage Through the Data Supply Chain

by Greg Aden

In a galaxy far, far removed from my current life as a data specialist, I approached books as a freight forwarder. I handled imports for Ten Speed Press (publisher of such classics as What Color Is Your Parachute?) and knew full well that a myriad potential problems lurked in production and transport. I also knew that proper practices and relationships were essential for success throughout the physical supply chain.

The same holds true for the data supply chain that you use to convey information to book wholesalers and retailers, and that they then use to present information about your books to customers on the Web. Data sells your books. It creates and invigorates your publishing process, allowing you to negotiate rights, present your product, and express your ideas. Given the power of these bits and bytes, it’s important that data be conveyed, displayed, stored, and, finally, shipped properly.

While humans are a responsible and integral part of this process, the systems they use must allow for accessible and malleable data that will enable productive trading relationships. Awareness and attention to the data chain will produce the perfect presentation of your book at many locations online.

Human Input That Brings Trouble

Problems often begin when people enter data in your source database. More specifically, they often begin with character translation. We’ve all seen gobbledygook on Web sites that results from faulty translation of characters, and most publishers have suffered through phone calls from authors whose listings have been affected.

Causes vary, but in general character-translation problems can be traced to three habits:

Cutting and pasting from MS Word. This process can lead to translation errors because Microsoft uses non-ASCII characters. Have you looked closely at the apostrophes and quotation marks created in Word? They are scripted. They are not ASCII (aka basic text). Therefore, they don’t translate well. Sometimes they translate as squares; other times they appear as question marks, or as a bold, vertical line, or “pipe.” There are several variations.

The solution is a thorough review (especially of the descriptive fields, TOC, Main Description, etc.) to make sure the non-ASCII characters have been stripped, even if it means dropping the offending text into an ASCII editor, like Textpad, and Find/Replacing those characters.

Using complex HTML. I highly encourage the use of HTML for descriptive text, but it’s important that it be basic HTML—<p> for paragraph, <br> for line break, <i> for italics, <b> for bold, <u> for underline, and <li> for bullet points. If you stick to those, you should be fine. Otherwise, approach at your own risk.

Remember, you need to provide a file that all booksellers, and especially e-tailers, can handle. If you don’t use <p> or <br> rather than carriage returns in your descriptive copy, some of that copy may get cut off. Different sites handle carriage returns in different ways.

Poor internal organization. Commonly, editors or IT specialists contribute information, and nobody with the necessary expertise reviews it. For both large and small publishers, this causes problems. Many publishers also offload the data issues to their distributors, which may have their own problems with data.

A system of oversight or commitment to a standard operating procedure is essential; otherwise bad data will continue to flow into the supply chain and interfere with sales.

About Timing

Barnes and Noble, Bowker, and Borders push for product information 180 days in advance of publication. For many publishers, this isn’t realistic. However, the message matters, because the sooner a publisher provides the information, the sooner both the publisher and the bookseller will begin to realize revenues. Just as you wouldn’t wait until December 24 to deliver active product to a store, you shouldn’t deliver your online product data at the last minute.

Ideally, publishers transmit the book metadata for prepub titles while updating price, availability, and descriptive copy on titles that have just become active.

Managing Conversion

Even if your entry of information is flawless and your timing is admirable, you can still be plagued by data glitches. Why? Because you need to convert your information to a format that your trading partners can accept. For most parties, it’s the XML-based standard called ONIX. For others, it’s a tab- or pipe-delimited file. For some, it’s both, and a few have entirely different requirements. (See “Use Better Data to Boost Your Sales” by Michael Healy in the May Independent for additional information, and to learn about Product Metadata Best Practices, which can be downloaded without charge from bisg.org/docs/Best_Practices_Document.pdf).

The main question here is: Do you have a database that can efficiently map your product information into different data formats, do it regularly, and (preferably) push out only the titles for which you’re supplying new information or changes, aka “delta” files?

Databases are not always flexible. Many publishers can’t update their title information easily, and as a result they find themselves with bad legacy data on their trading partners’ sites. In a perfect world, your database could pick, pack, and manipulate data like skilled warehouse workers. In the worst scenario, you can find yourself paralyzed by existing systems and charged back for bad data by e-tailers.

The solution is either a small army of people to regularly map your data to the correct format, or a software platform with a powerful engine that doesn’t require hours and hours of setup.

Distributing Data

The final hurdle is distribution. One large unnamed publisher will ship only to booksellers that can receive ONIX or an EDI transmission. Needless to say, that company is missing opportunities for sales.

In the physical shipping realm, your distribution channels may journey through the United States and Canada, but what of Asia, Europe, and South America? In the data world, your information may travel smoothly to Amazon, BN, Borders, and leading book wholesalers, but what about Thematic Attic, MBS Textbooks, Bookmanager, and many other steady and dependable booksellers and partners?

They’re an FTP transmission away. If you care to address this in-house, you can use one of many inexpensive FTP products on the market. One Web-based solution called YouSendIt (yousendit.com) is particularly easy for one-off FTP transmissions. Of course, the optimal solution is a script that transmits data to all partners in one fell swoop, and allows you to add new trading partners as your list grows. This is more complicated.

Why not form relationships with those partners previously mentioned, or others among the approximately 100 available parties looking for book data in North America alone? I suggest finding out whether there’s an untapped market that would work for you. Getting good data online requires no customs clearance and sets the lure for new client streams.


Your product’s online representation is as important as the physical jacket of the book. If it’s correct in content and format, you will attract new customers. If it’s delivered on time to a large spectrum of booksellers, you will increases sales.

Greg Aden is the CEO of NetRead and an original member of BISG’s ONIX steering committee. Large and small publishers use his Web-based product, JacketCaster, to convert and distribute title information and increase book sales. To learn more, emailHYPERLINK “mailto:graden@netread.com” graden@netread.com, or call 206/277-7886.



Connect With Us

1020 Manhattan Beach Blvd., Suite 204 Manhattan Beach, CA 90266
P: 310-546-1818 F: 310-546-3939 E: info@IBPA-online.org
© Independent Book Publishers Association