Jeff Lynch [MVP]

June 2008 - Posts

• Understanding RosettaNet Message Flow in BizTalk Server 2006

  If a picture is worth a thousand words, then these two diagrams (courtesy & copyright Microsoft) are worth millions! At least they are to anyone desperate to understand how the BizTalk Accelerator for RosettaNet (BTARN 3.5) really works. Initiator Message Flow Don’t let the complexity of this diagram scare you. The message flow for an outbound RNIF (or PIDX) message consists of four key areas. The SQL Server BTARN Databases where all “lob” messages (such as a P21 – PIDX Invoice) and any attachments are stored BEFORE the outbound RNIF message flow actually begins, and the SQL Receive Location used to process the message content and attachments into the MessageBox. Note: If you’re not sending attachments you may elect not to use SQL for this and can substitute FILE receive locations. The SDK contains sample code and a pretty good explanation of how this may work. The Initiator Private Process orchestration where the actual messages and attachments are “prepared” (usually transformed into XML) for further processing by the initiator public process. The Initiator Public Process orchestration where the outbound message is actually formatted to meet the RNIF standard. This includes adding the required RNIF headers and other such tasks. Consider this a “Black Box” since you can’t change how this works without invalidating the RosettaNet certification. The HTTP/HTTPS Send Ports and the RNIFSend.aspx web application that are CRITICAL to the correct processing of the outbound messages. This is where the real work is done to validate the messages, create the correct MIME headers and add the required RNIF DOCTYPE declarations. Just a side note. If you’re really interested in how this all works, you can look at the source code for this web application using Lutz Roeder’s Reflector. I think you’ll be amazed at how much Regex is used to “craft” these RNIF messages. Responder Message Flow The message flow for an inbound RNIF message (whether it’s a new message or an asynchronous response to a message you’ve already sent) consists of three key areas. The RNIFReceive.aspx web application which processes the inbound message, validates it and then removes all the RNIF specific headers and DOCTYPE declarations. This then passes the message along to the HTTP/HTTPS Receive Port and Pipeline which handles the non-repudiation, decodes the message, parses the message and resolves which Party it came from. The Responder Public Process receive the RNIF message from the MessageBox and extracts the content and attachments and sends them to the responder private process for further processing. Just like the initiator public process, the Responder Public Process should be considered a “Black Box” and left unchanged. The Responder Private Process then routes the inbound messages and any attachments to the SQL Server BTARN Databases where it can be further processed and sent to some “lob” back-end system. There are a few other details involved in the message flow (signals) depending upon whether your transmission is synchronous or asynchronous, but the basics are fairly straight-forward. The best thing is that this workflow is almost entirely “configured” using the BTARN Management Console and is “message agnostic”, meaning that you can use any standard RNIF, CIDX or PIDX “PIP” (Partner Interface Process) that you and your trading partner both support. Best of all, it’s included in the price of BizTalk Server 2006 R2! Currently listening to Carrie Underwood’s Carnival Ride Posted Jun 26 2008, 04:54 PM by jlynch with 3 comment(s) Filed under: BizTalk Server, Microsoft, PIDX, RosettaNet

• PIDX Schemas Solution for BizTalk Server 2006

  One of the things you’ll need when working with PIDX / RNIF / BTARN projects in BizTalk Server 2006 is the actual PIDX schemas (.xsd) to be used. Luckily the American Petroleum Institute (API) has a public web site for the Petroleum Industry Data Exchange (PIDX) committee which contains links to the various versions of their schemas. Once you and your trading partner decide on which version you’ll be using, you can download the schemas and add them to your BizTalk solution in Visual Studio. When you attempt to open these schemas you’ll notice an error message that pops up as shown here. This is a fairly common error and usually occurs when the schema you are attempting to open contains a reference to another “imported” or “included” schema and the BizTalk Schema Editor tool cannot find the file in question. The work-around for this is fairly simple and all you’ll need to do is change these two attributes. First you’ll need to click on the Imports attribute collection and delete the reference to the PIDX Library schema which contains the hard-coded location which is incorrect. Next you’ll need to import a new schema as an XSD Include and select the PIDXLib from the picker as shown here. Once that step is complete, you’ll also need to change the Root Reference attribute so that the BizTalk Schema Editor knows how to correctly identify your schema. And that's really all there is to it, except you'll have to do this for every PIDX schema and every version. Good thing I've already done this and you can download the BizTalk Solution containing all the PIDX schemas right here! Posted Jun 25 2008, 04:05 PM by jlynch with no comments Filed under: BizTalk Server, Microsoft, PIDX, Rosetta Net, RosettaNet

• BizTalk Server 2006 - BTARN 3.5 Configuration Tips

  Just a quick post to those of you planning to use the BizTalk Accelerator for RosettaNet 3.5. There are a few little “issues” that you need to know about before you get started. Accounts, Hosts & Host Instances BTARN 3.5 requires the use of Hosts that have the Authentication Trusted option enabled. This has to be configured BEFORE you create any Host Instances. I recommend creating a new Service Account, a new In-Process Host, a new Isolated Host and then a new Host Instance for each of these as shown in the table below. You will also need to create new HTTP and SQL Send Handlers and Receive Handlers as well. Note: The In-Process Host Instance account and Isolated Host Instance account must be the same for BTARN to work properly! Service Account: Domain\AccountName In-Process Host: RosettaNetApplication In-Process Host Instance: RosettaNetApplication Isolated Host: RosettaNetIsolatedHost Isolated Host Instance: RosettaNetIsolatedHost SQL Send Handler: RosettaNetApplication SQL Receive Handler: RosettaNetApplication HTTP Receive Handler: RosettaNetIsolatedHost HTTP Send Handler: RosettaNetApplication Installation & Configuration Don’t use “localhost” for your BizTalk Server or Web Server name when running the Configuration Wizard. Use the NetBIOS name of your BizTalk Server machine and the NetBIOS name of your Web Server machine instead. The Configuration Wizard will fail with a strange error if you use default “localhost”. Why? I have no earthly idea but I will be submitting this to the BizTalk team as a bug! One final note All the BTARN bits are deployed to the BizTalk Application 1 by default. I’m still trying to find a way to isolate the RosettaNet bits (schemas, maps, pipelines, ports and orchestrations) in their own BizTalk Apllication much like the new EDI/AS2 bits are. I’ll post if I find an easy way to accomplish this. Happy Integrations! Update I just want everyone to know that I'm not really all that smart. I had a ton of help setting up BTARN 3.5 and getting things working properly! I hired the folks from Microsoft Consulting Services to come in and teach me everything I could learn about using RosettaNet in BizTalk Server 2006. Special thanks to Ross Santee! A great consultant. Currently listening to: Dave Koz – Castle of Dreams Posted Jun 18 2008, 09:42 AM by jlynch with 6 comment(s) Filed under: BizTalk Server, PIDX, RosettaNet

• Tips for Using PIDX Schemas in BizTalk Server 2006

  Day 3 in my RosettaNet implementation nightmare and I still can’t see the end of the road! As you may or may not know, the American Petroleum Institute (API) has it’s own set of XML schemas and transport standards know as the Petroleum Industry Data Exchange or PIDX for short. This is the actual “standard” that my current RosettaNet project is really all about. The interesting about these schemas is the fact that they use their own unique namespace and prefix as shown below: xmlns:pidx=http://www.api.org/pidXML/1.0 Since the BizTalk Mapper tool always generates XML document with the “ns0″ prefix and there is no property you can set to change this, you’ll need to use a little “XSLT slight-of-hand” to get this to work. Step 1: Create your BizTalk Map as you normally would using the mapper tool. Step 2: Validate your map and open the xslt generated by BizTalk Server 2006 in your favorite text editor. This file can usually be found in the “\temp\_mapdata” folder. Step 3: Replace all instances of the namespace prefix “ns0″ with the prefix you require (which in this case is “pidx”) as shown below. Step 4: Save the result as an .xslt file and include it (or add it) into your BizTalk project. Step 5: Create a new map using the same source and destination schemas you used in Step 1 but use the external xslt you generated by setting the “Custom XSLT Path” property on your new map. The results are shown in the figure below. Viola! The “ns0″ namespace prefix has been replaced by the desired “pidx” prefix. Click below for more information about using external xslt in your maps. Custom XSLT in BizTalk Maps Take Away: There is always more than one way to skin a cat! In BTS2006 it may not be obvious how to do something a little unusual. Posted Jun 13 2008, 02:18 PM by jlynch with 2 comment(s) Filed under: BizTalk Server, PIDX, RosettaNet

• Business Process Automation – Don’t Just Shift the Costs!

  I'm stepping on my soapbox again so be warned! I've recently encountered a number of well-intentioned (or perhaps not so well-intentioned) companies looking to reduce their internal costs for processing orders and invoices through some sort of "business process automation" system. Being an avid BPA developer I applaud these efforts so long as they bring "value" to everyone in the supply chain. What I see all too often is companies spending thousands and thousands of dollars on web-based initiatives where the main goal is to shift the transaction processing from themselves to their suppliers. From a business perspective, this just isn't a wise choice and from a technological perspective, it just isn't necessary. Having your suppliers enter their invoices first on their ERP system and then on your web-based system does not reduce the "transaction cost" it just shifts the labor from yourself to your supplier. True B2B Integration Doesn't Just Shift Costs, It Eliminates Them! The Internet offers developers the unparalleled opportunity to connect disparate systems together without paying EDI "VAN" (Value Added Private Network) charges, telecom "WAN" (wide area network) charges or even leased-line charges. Basically, all the infrastructure costs are essentially free or paid for though corporate web access. All the developer (and business analyst) has to do is "connect-the-dots" using products like Microsoft’s BizTalk Server, Tibco’s iProcess Suite or Software AG’s WebMethods product to eliminate the real transaction costs. It's never been easier or more affordable to do this and the benefits to the entire supply chain are enormous! Automating the transaction processing so that the document (purchase order or invoice) is "touched" by human hands only during it's creation doesn't just shift costs, it eliminates them! Yes, I've heard all the arguments about how "costly" B2B integration can be and how smaller companies really can't afford to participate but IT JUST ISN'T TRUE! Products like BizTalk Server 2006 make BPA truly affordable for companies of any size. I know of at least one very small company that used this product to integrate QuickBooks (their "ERP" system) to their much larger suppliers and customers. If they can do this without breaking the bank, anyone can! Developer Call To Action: The next time your senior management talks about automating business processes with customers and suppliers, don't just develop a BPA system that shifts the costs, work to eliminate them altogether! Posted Jun 12 2008, 04:30 PM by jlynch with 5 comment(s) Filed under: B2B, BizTalk Server, Business Process Interation, Microsoft

• RosettaNet: Who Invents This Stuff?

  Captain Picard - “Admiral, we’ve engaged the Borg” Day 2 in my RosettaNet development project and all I can say is “Who Invents This Stuff”? Yes, it’s been a glorious two days of trying to comprehend the world’s most complicated business-to-business process. Only the electronics industry could conceive of something so overly complex. Secure, internet-based transactions were never meant to be this difficult. It’s like swatting a fly with an atom bomb! After ten years of B2B development I though I had seen it all. Boy, was I wrong! And the worse thing is that you have to pay for the privilege of enduring this torture. Every other xml standards organization I’ve ever dealt with freely distributes their schemas and specifications for everyone to use. Not so for the folks at RosettaNet.org! You’ll have to pay anywhere from several hundred to several thousand dollars for the rights to use their “PIP”s (Partner Interface Process) which is really nothing for than an XML DTD or XSD (schema) and a process guide explaining the message flow. It looks like RosettaNet.org used to publish these on their web site but now you’ll have to pay for a subscription to get these. Why on earth would so many smart B2B developers adopt such an overly complex, expensive and time consuming standard is beyond my understanding. Folks, business-to-business transactions do NOT have to be this difficult. Stay tuned for more fun & games! Posted Jun 11 2008, 09:24 AM by jlynch with 4 comment(s) Filed under: BizTalk Server, RosettaNet

• BizTalk Server 2006 R2: To Boldly Go Where No Man Has Gone Before!

  This week I’m going to embark on a dark and mysterious journey, fraught with great danger, both real and imagined. I’m about to begin development of a BizTalk Accelerator for RosettaNet (BTARN 3.5) project with the assistance of Microsoft Consulting Services. For those of you experienced in BizTalk development, you’ll understand my trepidation. For those of you familiar with RosettaNet (RNIF), you’ll understand my sheer terror! If I survive this ordeal, I promise to post about my experiences, so that future generations of BizTalk developers don’t make the same mistakes, don’t fall into the same traps and don’t lose what little is left of their rapidly graying hair! Stay tuned! Posted Jun 09 2008, 09:53 AM by jlynch with 10 comment(s) Filed under: BizTalk Server, RosettaNet