Portfolio and Technology Blog: 20 Years of Tinkering

I've come across an issue several times when I try to backup a VEagent backup host itself (aka proxy host).  After some back and forth troubleshooting with HP Software Support, we finally determined that this is a known VMware limitation.  If the proxy host itself is a virtual machine, and the transport method to back it up is HotAdd (only option in Data Protector), then a backup job will result in an extra virtual disk being left over on the proxy host.  This will eventually, if not immediately, leave the host corrupted.  And once the proxy host is corrupted, any future VM backups that rely on it will not run.  Not good!

The only work-around is to NOT backup the proxy host itself using Data Protector, or to make the proxy host a physical machine.  More details on this issue can be found on the VMware support site.

In the event that this scenario has already played out in your environment, you currently have a VEagent client in Data Protector that is inaccessible and unusable, and all other VE backup jobs dont run as a result.  To fix, we do the following.

1. Create a new VM to serve as a new VEagent backup host.  And add it as a VEagent client in DP GUI.
2. Normally, you can right click a client in DP to delete it, but if the old VEagent host is unreachable, this will not work.  To remove this old client, delete the related line in the cell_info file on the DP cell manager.  This is found in: "C:\ProgramData\OmniBack\Config\Server\cell" (server 2008).  This will remove the old host from the DP GUI going forward.
3. Lastly, you're existing VE backup jobs are still going to be referencing the old proxy host.  The old proxy hostname is hardcoded in the various VE barlist files for each of your VE backup jobs.  Open each of these files and swap out the old hostname with your new hostname.  These VE barlist files can be found on the cell manger in: "C:\ProgramData\OmniBack\Config\Server\BarLists\VEAgent" (server 2008).

You're now setup to move forward using a new VEagent backup host.

UPDATE:
There is now a better fix to this issue.  Please see this post:

The goal here is to create a SharePoint form that contains several fields that will be filtered based off previous selections in the form. In the example that follows starting with a Category, only specifc Vendors will be available, likewise for Topics available for selected Vendor, and so on for Duration, Description, and Instructor. What we want to do is filter the available options in the form as selections are being made.

Here is what the final result will look like:

Setup SharePoint Lists

So lets get to it. Start with 2 sharePoint lists, my example will use the following:

• Training Catalog
• Training Request

The Training Catalog list contains the raw list of available trainings and uses the following columns:

• Category
• Vendor
• Topic (Rename the title field to Topic)
• Duration
• Description
• Instructor

Note: these columns are all single line of text. Lookups or other column types may work, but are untested.

The Training Request list contains the exact same columns as above. Other fields, such as "When" can be included as well, but have no impact on this tutorial either way.

Note: These columns MUST be created as single line of text.

Open SharePoint Form in InfoPath

Open the Training Request list form:

• List Tools > List > Customize Form, or
• List Settings > Form Settings > Modify the existing InfoPath form > OK.

After the form loads in InfoPath, select the first colum (Category) and right click it > Change Control > Drop-Down List Box.
Do the same for Vendor, and Topic.  Filtered data can only be applied to specific Control Types. Drop down fields are ideal for our use case here.
And we will only need to filter data on these first 3 fields, the rest can remain as Text Box controls.

Create InfoPath Data Source

Field: Category

Right Click Category again and select Drop-Down List Box Properties.
Select Get Choice from an external data source, and to the right of Data Source select Add…

1. In the Data Connection Wizard Create a new connection to: Receive data > Next.
2. Select the source of your data: SharePoint library or list > Next.
3. SharePoint site details: the location of the SharePoint site should be prepopulated > Next.
4. Select a list of library: Training Catalog > Next.
5. Select Fields: Category, Vendor, Topic, Description, Duration, Instructor, and any other fields you might have included in your Training Catalog list.
6. Sort by: Categoy > Next > Next > Name your new Data Connection and select > Finish.

Back in the Drop-Down List Box Properties dialog box, the Data source field should now point to your new data connection, in my case named Training Catalog. 

• To the right of the the Entries field, click on the Select XPath button.
• In the Select a Field or Group dialoag box, select d:SharePointListItem_RW and press OK.

Note: The Category column does not need to be filtered as it is the first field in our form.

• Back in the Drop-Down List Box Properties dialog box set the Value and Display name fields both to d:Category.
• And for good measure, check the Show only entries with unique display names and click OK.

Setup Some InfoPath Filters

Field: Vendor

• Right click Vendor field and select Get choices from an external data source and set the Data source to Training Catalog.
• Click the Select XPath button to right of the Entries field.
• In the Select a Field or Group dialoag box, select d:SharePointListItem_RW.

Select the Filter Data… button in the Select a Field or Group dialog box and In the Filter Data dialog box click the Add… button:

• In the first drop down field choose select a field or group.
• The Data source should be set to Training Catalog (Secondary). Choose "Show Advanced view" if you cant see the Data source.
• Under d:SharePointListItem_RW select Category and then press OK.
• The middle drop down qualifier should be set to "is equal to".
• In the third drop down field choose select a field or group, select the Show advanced view, and set the Data Source to Main.
• Then, under the my:SharePointListItem_RW group select Category and click OK four times to close out the various dialog boxes.
• Finally, set the Value and Display name fields both to d:Vendor and check the Show only entries with unique display names.

Field: Topic

• Right click Topic field and select Get choices from an external data source and set the Data source to Training Catalog.
• Click the Select XPath button to right of the Entries field.
• In the Select a Field or Group dialoag box, select d:SharePointListItem_RW.

Select the Filter Data… button in the Select a Field or Group dialog box and In the Filter Data dialog box click the Add… button:

First Condition

• In the first drop down field choose select a field or group.
• The Data source should be set to Training Catalog (Secondary). Choose "Show Advanced view" if you cant see the Data source.
• Under d:SharePointListItem_RW select Category and then press OK.
• The middle drop down qualifier should be set to "is equal to".
• In the third drop down field choose select a field or group, select the Show advanced view, and set the Data Source to Main.
• Then, under the my:SharePointListItem_RW group select Category and then select the And button to add another condition.

Second Condition

• In the first drop down field choose select a field or group.
• The Data source should be set to Training Catalog (Secondary).
• Under d:SharePointListItem_RW select Vendor and then press OK.
• The middle drop down qualifier should be set to "is equal to".
• In the third drop down field choose select a field or group, select the Show advanced view, and set the Data Source to Main.
• Then, under the my:SharePointListItem_RW group select Vendor and click OK four times to close out the various dialog boxes.
• Finally, set the Value and Display name fields both to d:Topic and check the Show only entries with unique display names.

If you preview the form in InfoPath, the first 2nd and 3rd fields should now only display relevant data based on the 1st and 2nd field filtering. Close the preview when you're finished.

For the final 3 fields, there are no "choices" to make. And thus there will be no more fields to setup filters on.
We simply want to populate the Duration, Description, and Instructor fields.

Set Field Values using InfoPath Actions

1. With the Topic field selected, choose Home from the top menu and select Manage Rules. In the Rules Pane, name your new Rule.
2. Under Condition: a default rule should be created "None – Rule runs when field changes" – leave it at that.
3. Select the Add button under Rule Type: Action and choose "Set a fields value".

We will need to define 3 actions, one for each of the remaining fields (Duration, Description, Instructor).

In the Rule Details dialog box:

• Set the Field to (Main) Duration by clicking the XPath button (remember to "Show Advanced View" to select Main data source).
• Set the Value by clicking the fx button, select Insert Field or Group… and choose (Training Catalog (Secondary)) Duration field.
• Before you hit OK, additionally select Filter Data > Add to setup some filter conditions.

In the Specify Filter Conditions dialog box, do as we did before and set your conditions for Domain, Vendor, and this time include a Topic condition as well.

The Specify Filter Conditions should be as follows:

• (Training Catalog (Secondary)) Category "is equal to" (Main) Category "and"
• (Training Catalog (Secondary)) Vendor "is equal to" (Main) Vendor "and"
• (Training Catalog (Secondary)) Topic "is equal to" (Main) Topic

Select OK > OK > OK > OK to close out all the dialog boxes.
And finally, the Value box should look like this: Duration[Category = Category and Vendor = Vendor and Topic = Topic]
Click OK.

Repeat these steps for Description and Instructor by adding 2 additional actions to the Rule created on the Topic field. If you preview the form in InfoPath now, the first 3 fields will be filtered, and the final 3 fields will be populated based on the preceeding 3 fields.

And that's it!!!  

What follows is the recommended information and logs one should gather when opening a Data Protector case with HP.  Note: the following only applies to a Data Protector 7.01 Windows Environment.

1. Server Specs 
2. Data Protector Patch Level
3. Full Session Report
4. Extended logs

1) Server Specs

No matter the issue you are having, invariably the server specs for the Cell Manager need to be provided.  This may also apply to your Installation Servers, Backup Host, or any other affected hosts by the issue you are having.  Generally the OS/Build is acceptable, such as: Windows 2008 R2 Enterprise x64.

2) Data Protector Patch Level

As with providing server specs, the patch level of all related systems will usually be requested as well.  Run this command from every host that has the DP inet agent installed:

omnicheck -patches

Copy/Paste the output into the support case for each system related to the issue.

3) Full Session Report

More than likely you will want to include the full session report from the backup job that had issues.  This session report will contain a detailed output of the backup job containing all of the related messages you would see in the DP GUI.  You can find the Session ID by browsing the Reporting context in DP GUI, and selecting your job.  The right pane will look something like this:

[138:742] Backup session "2013/07/22-3" of the backup specification "VEAgent VM-Desktops",backup group "VMWARE" has errors: 19.

In this example, 2013/07/22-3 is the Session ID. Copy/Paste that value into the following cmd, which you will execute on the Cell Manager:

omnidb -session <sessionid> -report > C:\session.txt

Upload the session.txt file to HP Support case.

4) Extended logs

In many cases, the default session report will not provide enough debug information to find what is actually the problem.  What will be needed is some extended logging.  However, to gather this additional debug info, you will have to first enable the debugging, re-produce the issue, and then disable the debugging.  The debug process will typically create dozens of log files, depending on the issue, which you can then zip up and upload to the HP Support case.  

There are two primary debug use cases:

1. Troubleshooting Data Protector GUI
2. Troubleshooting DP Backup Jobs that are failing/etc.

Troubleshooting DP GUI

Exit the GUI, and restart it from the MS-DOS prompt in debug mode:

cd \Program Files\Omniback\bin
manager.exe –debug 1-500 yourname.txt

Reproduce the error in the DP GUI and then exit the GUI to stop the debugging.  Depending on the nature of the issue, this will create a debug.txt file on every host related to the issue.  On each host, in the following location, look for long file names starting with OB2DBG and ending with yourname.txt:

(Windows 2003) Program Files\Omniback\tmp        
(Windows 2008) Program Data\Omniback\tmp

Gather all these debug files from each host, zip them up (per host), and upload to HP Support Case.

Troubleshooting Failing Backup Jobs

Depending on the nature of your failing backup job you might deviate from these steps, but this will generally apply to all failing backup jobs:

On the Cell Manager open the CMD prompt and execute the following:

omnisv stop / omnisv start -debug 1-500 yourname.txt

On any other related system(s) with a DP INET agent installed, go to Windows services, stop the Data Protector INET service.  In the start up parameters for the service add the following and then click start:

-debug 1-500 yourname.txt

Reproduce the backup job failure, and exit the DP GUI.  This will create dozens of logs on the Cell Manager and every related system debugging was enabled on.  Before you collect the logs however, be sure to disable the debugging by removing the service startup parameters you added to each system and restart them normally.  Also, don't forget to restart DP Cell Manager without the debug:

omnisv stop / omnisv start

On each host, in the following location, look for long file names starting with OB2DBG and ending with yourname.txt:

(Windows 2003) Program Files\Omniback\tmp        
(Windows 2008) Program Data\Omniback\tmp

Gather all these debug files from each host, zip them up (per host), and upload to HP Support Case.

Finally, you will also want to grab a copy of the cell_info file from the Cell Manager and upload it as well:

(Windows 2003) Program Files\Omniback\Config\Server\cell    
(Windows 2008) ProgramData\Omniback\Config\Server\cell 

I've been bashing my head against the wall trying to understand why custom HTML used in a SharePoint Workflow "Send an Email" Action looked so awful when delivered.  In short, there seemed to be many more carriage returns, or new lines, than there should have been.  I was also having issues getting table background colors to render, and forcing externally referenced images to show.  

Inserting Custom HTML into Workflow Emails

Well, the good and bad news was that the answer was right under my nose the entire time.  Here's how I tracked down the last 2 issues mentioned above.  To add a background color to a table, insert an image, or implement any other custom HTML, you need to insert the raw HTML into the Advanced Properties window of the Send An Email action (reference).  Simply highlight the Email Action in your workflow and from the Top Ribbon select Advanced Properties > Body > and "…".  From there, you can insert HTML directly into the body of your email.  And it works perfectly, unless you are forgetting one simple thing.

"Optimizing" Workflow Emails 

SharePoint likes to handle HTML very "differently" then most applications.  That goes for all other Microsoft products frankly, and HTML in Workflow emails is no exception.  The key point to remember is that Workflow Emails containing HTML are expecting everything to be in one long string (reference).  No line breaks or carriage returns between any of your tags can be present.  Technically, this does shave a few hundred bytes (or maybe even a KB) off the document size, and could thus be considered "optimized."  However, the main issue here is that we are already jumping through so many hoops when working with the SPD String Builder (i.e. the Body editor of an email):  Copy/Paste is not possible, for instance, unless all "Lookups" have been removed.  By forcing HTML to be one long string we now introduce another issue where the HTML becomes nearly unreadable to human eyes should troubleshooting or future modifications be required.

Just to verify that indeed SharePoint expects HTML to be one long string, we can do the following.  Create a new "Send an Email" action and include some generic text, without using html.  Something like this:

If we then use the steps above to access the Advanced Properties of the Send an Email action, we will see exactly what has occurred behind the scenes:

Thus, we need to follow suit if we wish for our emails to render even remotely close to what our markup dictates. Seriously though, I assumed the DTD for SPD 2010 was HTML 4.0, but HTML 3.2??? OMG! These days, most modern Email Clients will accept at least XHTML 4.01 Transitional, and that is the markup DTD that I am now using for all my SPD 2010 Workflow emails.

Final Note on SPD 2010 "Optimization"

Back in the days of SPD 2007, it was really easy to Optimize HTML for pages that were not connected to a SharePoint site.  You could just go File > New > HTML Page, paste in your HTML, and Optimize it.  This really elevated SPD from being strictly a SharePoint tool to a handy Webmaster tool overall.  The Optimize functionality still exists in SPD 2010 – its buried in the menu Ribbon under Spelling > Optimize HTML > Remove Whitespace > HTML all whitespace.  However, by the nature of SPD 2010, this feature can only be used on a page that "lives" in a SharePoint site.  So in theory you could create a document library that contains workflow HTML templates, in a human readable form, and whenever you need to Copy/Paste the HTML into a workflow, just temporarily run the Optimize HTML function against your page, Copy/Paste the "Optimized" HTML into a workflow, and close the HTML page in SPD without saving it.  While this isn't a terrible idea, it does seem counter-intuitive and clunky in some ways, but maybe it's just me being difficult.

WARNING: 3DRipperDX has been reported by several sites as containing a virus/malware. Wether it is a false positive or a real threat, I am not sure. Proceed with caution.

In my last post, I described how to export geometry from Google Earth into an OBJ file, using GLIntercept.  And despite the lengthy setup instructions, that is a solid approach to capturing geometry from OpenGL applications.  However, there is no direct way to export the geometry with the texture data/coordinates mapped automatically.  For my purposes, the goal remains to export a city from Google Earth into a modeling application.  And while Google Earth does NOT have stellar textures by any stretch, it would be nice to have these included in the exported model.

3D Ripper DX (link removed, see warning above) seems to do exactly that, and the setup time is incredibly faster.  In just a few minutes I was able to follow this video tutorial to capture, export, and import an entire neighborhood, with textures, into 3D Studio Max.

As you can see the textures aren't really that great, but it is perhaps better than none at all!  Also, based on the instructions given in the video tutorial, I might be able to capture crisper textures if I play around with it a bit more.  The only other downside (for me) is not knowing how to use 3D Studio Max, at all, but that is obviously a personal issue.

Up next?  Exporting this bad boy into UDK.  🙂

Here is another video that you may also find useful by Paul Fatkins:

See these updated posts for more info:

Part 1 – Exporting Google Earth Models into UDK

Part 2 – Exporting Google Earth Models into UDK