Paul Kiel's Data Integration Blog
Data integration using Xml / Xslt and anything else...







Thursday, April 16, 2009

Fourth data standards consortia

I've been talking directly with my 4th data standards consortia (all listed below). It seems I am getting a reputation for this. The fourth is a new organization that is dedicated to world food traceability. Recent news stories about problems have illustrated the need to find a solution. News reports of Peanut butter in Minnesota and tomatoes in the southwest show an outbreak of salmonella. But finding the source of these outbreaks has proven difficult. The folks at World Food Trace are working to create an effective solution that is very cost effective.
And in fact, my work with OAGi has begun to branch out because of its role as a platform on which other standards can build. So a convergence is occurring, which is connecting me to groups like CIDX, UN/CEFACT, STAR, AIAG, ISA, WBF, PDES (STEP) and others.

See 2007 post archives.

Monday, February 23, 2009

XBRL, HL7 and the rise of domain data models

Been reading several items relating to the new proposal to require the use of XBRL for financial reporting. Indeed, Kurt Cagle has an extensive post, saying "this should be interesting." He suggests that this development should have a number of effects, including an increased need for Xml skills. Even Alexander Falk of XmlSpy (actually the company name is Altova but their flagship app is xmlspy) has commented on his company's product native support for both XBRL and HL7.

While I haven't worked intimately with XBRL, I have looked at their library from a design / best practice perspective as it relates to the other Xml Consortia for whom I have consulted. Its ontological nature is interesting and I think has something to say about that kind of data model.
The reason I group HL7 in there is because of the new administration's effort to push for electronic records in health care. This doesn't name HL7 by name, but one can't help but connect the two. Now having worked with a number of Xml Consortia, from human resources to supply chain management to media to research administration, I have always been amazed at the contrast between how technologically advanced the product is (medical care) and how *under*-automated the infrastructure is.

So I'll end the post by saying that I hope the trend is for the better. There is so much really good work being done in these domain Consortia and I'd love to see them have a broader impact. Yes, they each have their quirks (and not all of them are worth implementing), but the promise of domain data models has always been good. And there are many of us out there who are working to make these Consortia adopt common design patterns. Now we need to roll up our shirt sleeves and make them more of a reality.

See 2007 post archives.

Tuesday, February 3, 2009

Schema Lightener 2.0 released

Announcing a new release of the “Xml Schema Lightener” utility. It has been completely updated and is now version 2.0.

What it does
----------------
It takes an Xml Schema and creates a subset based on user input. Some refer to this with terms such as a “subset Schema view”, “pruned Schema”, “LiteBOD”, or “profiled Schema”.

Why it does this
----------------
Consider these use cases:
  1. You have an Xml Schema and want to implement a subset (a selected portion) of the data points. Or you want to communicate the selected data points to an audience.
  2. You have an internal data model in Schema and want to communicate to a trading partner only the data points necessary for a particular integration.
  3. You are implementing an Xml Schema acquired from somewhere else (i.e. from a trading partner, or from a standards consortium such as OAGi, HR-XML, etc.) and want to implement only the data points relevant to your business context.
In all these cases, it would be useful to create a Schema that shows these points and can be used for validation. The Xml Schema Lightener allows you to create the contextual Schema that each of these require. It does it using simple XSLT and Xml. And a GUI is included to make it easy to use.

It has been heavily tested, includes a free flattener, and also has a simple GUI too.

Cost effective
----------------
This utility is presented with its full functionality in a very inexpensive package. The $29 (USD) price does not cover the full cost of development and is simply a way to offset a portion of its creation and keep it going.

For more information
----------------
See http://www.xmlhelpline.com/tools/ for more information and FAQ.

See 2007 post archives.

the joys of air travel

Cost of checked bag: $25

Cost of "emergency" exit seat: $30

In flight meal: $5

Bottle of water: $2

Airline customer service: priceless

See 2007 post archives.

Sunday, December 21, 2008

Solved! internal_power_error BSOD on hibernate

Strange title for the posting I know, but if you are having the same problem you'll recognize it. I've had this intermittent problem with hibernation and done a lot of research around it. It all began when HP replaced my motherboard because of a recall issue. It came back in good working order. However, whenever I tried to hibernate (on vista), it caused a BSOD (blue screen of death) and automatically rebooted. Not 100% of the time but the vast majority of the time.

I want hibernate because the laptop is in my bedroom and sleep mode allows a flashing blue light to continue indefinitely. At night, I don't want this flashing, so hibernate is perfect for my needs.

Did a bunch of research on this problem, which lots of other folks have had. Many said there was a process that somehow caused the machine to come out of hibernate immediately. Others said it was a BIOS issue. Still others said completely deactivate hibernation and then reactivate. The more adventurous of you can read more here, here, here, here, here, and here (even though I have 32 bit).

Here are some of the things I've tried:
Tried powercfg.exe /hibernate off and then back on. No effect. On power settings, completely disabled hibernate on all modes (power button, lid close, etc), then reactivated. No fix. I went into device manager and made sure none of the devices were "allowed to wake computer". No fix. Also checked task scheduler and removed any task that I thought could be responsible for waking the machine at the wrong time. Again no fix.

Someone suggested a corrupt hibernation file. Trashed it and had it create another. Not a fix. My BIOS is completely updated, so I didn't suspect that. And I didn't want to regress to an earlier version. (Also someone said this was fixed in SP1, but I already had that installed.)

So my solution came in this form. First, I disabled automatic restart. This allowed me to see the BSOD message before it rebooted. The error was "internal_power_error", which is of course horribly unclear. Here is how to disable auto restart from a help file:
1. Click on Start and then Control Panel.
2. Click on the System and Maintenance link.
3. Click on the System link.
4. In the task pane on the left, click the Advanced system settings link.
5. Locate the Startup and Recovery area and click on the Settings button.
6. In the Startup and Recovery window, locate and uncheck the check box next to Automatically restart.
7. Click OK in the Startup and Recovery window.
8. Click OK in the System Properties window.
9. You can now close the System window.
10. From now on, when a problem causes a BSOD or another major error that halts the system, the PC will not automatically reboot. Rebooting manually will be necessary.

Second, on my NVIDIA nForce Networking Controller (the culprit), I had previously made sure it was not allowed to wake computer. (Unchecked "Allow this device to wake computer" in power management tab.) The problem was that my system was trying to turn the device off to save power. In device manager, I "unchecked" the "Allow the computer to turn off this device to save power" box (see pic below, which has it checked as I found it originally). This seems to have fixed the problem. Not sure exactly why this was a problem, but I am happy nevertheless. I've been able to hibernate for a few days now with no errors. Hope this helps some of you out there with the same problem.

See 2007 post archives.

Wednesday, December 10, 2008

SchemaLightener gets a user interface

The Xml SchemaLightener now has a user interface. So making a thinner view of an Xml Schema is now as simple as point and click.
Screenshot (click to enlarge - image is a bit fuzzy):



The SchemaLightener can be integrated into other tools and so does not *require* the use of the UI. But the UI provides a nice easy way to perform this function. Need a quick view of the schema for a group meeting? No problem. Just point and click, selecting an Xml Schema to lighten, an Xml instance that is representative of what you want, and then executing the transformation.

This UI is built with Java, so it works cross platform. It has been tested on windows vista, xp, and linux (ubuntu). And as I said before, has been tested using many Consortia standard schemas.

Plus, the Flattener, which was recently added, can also be invoked using the UI.

See 2007 post archives.

Wednesday, November 19, 2008

SchemaLightener gets a Flattener

I've added a new feature to the Xml SchemaLightener, namely that of a Flattener. First, here are some definitions:

Lighten: to remove elements and types from a schema, while the remaining elements and types are still consistent with the original schema. To "prune", or create a subset of the schema. To create a "view" of the schema, focusing on certain components.

Flatten: to merge all schema dependencies connected via xsd:include into a single schema file. To create a "stand alone" version of a set of schemas. All components are consolidated into one file without any editing or changes. Does not pertain to dependencies connected via xsd:import (because this will not result in a valid schema).

I've often needed a flattening tool. Schemas are usually developed modularly to maximize reuse. This has tremendous benefits in building a data model. However, come deployment time, tools sometimes have a problem managing a web of xsd:include files. So it is generally recommended to create a runtime version of a schema which consolidates includes into a "stand alone" one. (Of course it may not literally be stand alone because of xsd:import files, but it isn't possible to merge these according to the spec.)

The difference between xsd:include and xsd:import is in namespaces. Files connected via xsd:include have the same default namespace. This is what allows them to be consolidated into a single schema. Files connected via xsd:import have different default namespaces.

So I've thrown in an XSLT which will create a stand alone or flattened schema into the SchemaLightener tool. Importantly, even though imports cannot be merged, it does automatically go to each import and flatten its includes (if any) and so the result is a fully valid set of schemas that may have imports but no includes.

In short, the absolute minimum number of schema files possible.

See 2007 post archives.

© Copyright Paul Kiel.

Archives:
January 2008 February 2008 March 2008 April 2008 May 2008 June 2008 July 2008 September 2008 October 2008 November 2008 December 2008 February 2009 April 2009
2007