Portfolio and Technology Blog: 20 Years of Tinkering

1. Install SiteScope 11.30
2. Extract "Software_HP_Operations_Agent_v11.14_Windows_ISO_TC097-15046" iso locally on sis server
3. CD to that extracted dir (run as admin)
4. Run "cscript oainstall.vbs -i -a" and click ok to any prompts, etc.
5. Select Start > All Programs > HP SiteScope > Configuration Tool (run as admin)
6. In the "Configure HP Operations Agent installed separately" option select "Configure HP Operations Agent" to complete the installation of the HP Operations agent.
7. Restart SiteScope server.
8. In SIS UI, select Preferences > Integration Preferences:
   • Click the "New Integration" button and select "HP Operations Manager Integration".
   • Configure the OM/BSM path (in distributed environment, point to gateway server)
   • Click "Connect" to connect the agent to the BSM server. This sends a connection request from the agent to the specified server.
   • Leave this window open
9. Accept the agent connection request on the BSM Gateway/Web Processing server
   • In BSM 9.20 or later UI, go to Admin > Operations Manager > Certificate Request, locate and grant your certificate request.
10. Install the log policies on the SiteScope server
    • After the certificate request has been granted on the BSM server, back in SIS UI, click "Install Policies" in the HP Operations Manager Integrations Main Settings panel of the HP Operations Manager Integration dialog box. This installs and signs the preconfigured log file policy file on the HP Operations agent.
    • Leave this window open
11. Check connection status and send test message from the SiteScope server – optional
    • In the HP Operations Manager Integration dialog box, expand the HP Operations Manager Integrations Main Settings panel, and click "Analyze".
    • To check that the agent is connected to the BSM server, type a message in the Test message text box, and click "Send Test Message".
    • If the test is successful, the text message is displayed in the Operations Management Event Browser in BSM. This message is generated by a basic opcmsg policy command (i.e. opcmsg a=a msg_grp=test o=o msg_t=test).
12. Enable SiteScope to send events to HPOM or Operations Management
    • In the HP Operations Manager Integration dialog box, expand the HP Operations Manager Integrations Main Settings panel, and in the Configuration Settings area select "Enable sending events".
    • Leave all other options at default
    • Save and close the window – done.
13. Note: LWSSO will automatically be configured between BSM and SIS.

In the last post on this subject I discussed an issue where backing up a virtual VEAgent/Backup/Proxy host itself leads to orphaned disks on the host (when using a HotAdd backup transport method), which inevitably leads to the host itself becoming unstable and crashing – thus causing any subsequent VM backups relying on he backup host to fail!

The workaround was simply to not run backups against your virtual VEAgent host, or to make your VEAgent host physical.  HP has since come back to me with an actual fix.

As of version 8.00, Data Protector uses VDDK 5.0U1, and our issue is related to the following Known VMware Issue:

http://www.vmware.com/support/developer/vddk/VDDK-510-ReleaseNotes.html
 
Hang in connect or cleanup due to intermittent race condition. 
Windows processes can sometimes hang when calling VixDiskLib_ConnectEx() or VixDiskLib_Cleanup() due to a race condition while loading or unloading libraries. There is no known workaround. A fix has been identified, and will be available in the VDDK 5.1 first update release
 
If this is the case then it will be fixed in a newer version of the VDDK, the reason why I suspect it might be related is because if the Vmware VixDiskLib_Cleanup() hangs then the disks will not be removed from the backup host.  During a HotAdd backup a redo log is created for HotAdded disks, so in this case HotAdd failed to properly clean up redo logs and left the VM disks in your backup host.

The Fix:

1. From the Backup host: remove all the folders inside of: C:\Windows\TEMP\vmware-SYSTEM
2. Then remove the orphaned disks on the host corresponding to other virtual machines in the backup host using vCenter (not host disk management).  After this, reboot the host.
3. If you have a related case open with HP, you can optionally add the variable: vixDiskLib.transport.LogLevel="6" to "C:\ProgramData\OmniBack\Config\client\ vepa_vddk.config".  This variable will log more information about the vddk transport method used during the backup job and this is the only way to determine why the HotAdd doesn’t remove the disks from the backup host.  Once the issue is duplicated with vixDiskLib.transport.LogLevel="6", collect the session report when the disk was not removed so we can get the information needed to determine the reason of this condition. 
4. On backup host, open cmd prompt and type following:

• diskpart
• san policy=onlineAll 
• Note: As per Vmware documentation, setting the san policy thus is a best practice when using HotAdd transport method.

In my last post on exporting Google Earth models into UDK, I linked to a video that covers the basic steps to export models from Google Earth into 3DSMax, using 3D Ripper DX.  Once you have the GE models in 3DS Max, complete with textures, its simple enough to export the scene as a .fbx file into UDK.  After you've worked through the process a few times, its pretty straight forward and easy to execute.  However, there are still some limitations.

1. While the textures render okay in 3DS Max, their .dds format is not acceptable for UDK.  So, these textures have to be batch converted to an acceptable format (i.e. tga) if you wish to include them in UDK with your .fbx model.  This can be done using IrFanView Batch converter, additionally utilizing the "All Plugins" plugin to recognize .dds filetype. After IrFanView and the plugin have been installed, proceed to this video (starting around 1:15) to setup and use IrFanView to bach convert your files properly for usage in UDK.  The only exception here is that in the video he is convering .jpg textures exported from Google Sketchup.  Simply swap this workflow out for .dds textures exported from Google Earth.  The rest of the steps will be the same for our uses.  After the batch conversion is complete, you should now have 1 .tga texture for every .dds texture.  In some cases this could easily be thousands of files.  Here is where manual work comes into play.  Every tga texture needs to be imported into UDK, with a material created, and manually assigned to your model, in place of the current .dds texture assignment.  And while this is possible, it is extremely tedious if your scene has anything more than a dozen buildings, which could easily be hundreds or thousands of textures…  I went through this, and it is painful.

Luckily, a reader of this site has been working on simplfying the entire workflow, and has thus far come up with the following:

1. Capture with 3D Ripper DX
2. Using IrfanView’s batch conversion, convert all the .dds files in the Textures folder to .png (or .tga, or any other texture file type UDK will accept)
3. Using a text editor, open the .mtl file in the Frames folder. Use find and replace to replace all “.dds” with “.png” or whatever file type you converted the dds files to
4. Using a text editor, open the .obj file in the Frames folder. At the very top, type "mtllib (*FILENAME*).MTL", where *FILENAME* is whatever the filename of your .mtl file is
5. Move all .obj, .mtl, and all the .png files into the same folder
6. In 3DS Max, import the OBJ file.
7. One, possibly two, .png files will be missing. Hit skip both times.
8. Run the MaxScriptForMakingMaterialsShowInViewport.ms script (included in this zip).
9. Run the MaxScriptForAdjustingSpecularHighlights.ms script (included in this zip).
10. Run the MaxScriptForCloningAndAddingUCX.ms script (included in this zip).
11. Export the scene as a .fbx file. Make sure the the Embed Media checkbox in the fbx export options is checked.
12. In UDK, import the .fbx file. Make sure that Import Materials in the import dialogue options is checked.

This simple workflow saves so much time, its not even funny!  And as a bonus, the models will now even have rough collisions in UDK.

There is one other final thing to consider, Scale.

Setting the correct scale in 3DS Max for use in UDK

Assuming we want our Google earth models in UDK to be as close to reality as possible (in scale), we need to adjust the Unit settings in 3DS Max to match the Unit settings in UDK.  That way if we know a building is 12' tall in Maxm it will also be 12' tall in UDK.

Create dummy box in Create Pane to represent a 6’ tall person

 
Select Modify pane, and set the following:

• Length: 32
• Width: 32
• Height: 96

Update Max Unit system settings to map to UDK scale

• Customize > Units Setup > US Standard > Feet w/Decimal Inches > Inches 
• In same dialog, System Unit Setup > in the “1Unit =” field press CTRL+N to open calculator.
• Input 12/16 (12” divided by 16 Max units).  This equals 0.75.
• Paste value into the 1 Unit = field.

Note: 3DS Max will not save these settings by default, and they will have to be re-applied everytime you open a new scene file.  Unless: 

• Create a new scene, apply the units setup above, save the scene as "maxstart.max" to "MyDocuments>3dswMax>Scenes"
• This will start max every time with this same scene. 
• You can use this for units setup or anything esle like lights, geometry, etc.

This will translate 96 Units into 6’0.0”

Now that you have a proper reference, and a matching unit scale from 3DS Max to UDK, scale your captured model to match the scale of the 6' block.  In general, I found that scaling the captured model up 3 separate times (by 100000, by 100000, and by 62000) worked for me as a close enough approximation.

I had a scenario where I needed to filter a SharePoint 2010 discussion list to only show discussions that had been updated within the last 60 days.  My Discussion list is inserted on a page using a DataViewWebPart (DVWP).  So, to filter the discussion list as required, select the DVWP, and then create a filter, like so:

Next we need to tweak the code of the filter so that it will be "Greater Than Or Equal to Today, minus 60."

In SharePoint Designer Code view, locate the following bit of code that resides towards the end of the DVWP:

...
selectcommand="<View><Query><OrderBy><FieldRef Name="DiscussionLastUpdated"
Ascending="TRUE"/></OrderBy><Where><Leq><FieldRef Name="
DiscussionLastUpdated"/><Value Type="
DateTime"><Today/></Value></Leq></Where></Query></View>"
...

Before we tweak this code, we can safely convert it as follows:

...
selectcommand="<View><Query>
<OrderBy><FieldRef Name='DiscussionLastUpdated' Ascending='TRUE'/></OrderBy>
<Where><Leq><FieldRef Name='DiscussionLastUpdated'/><Value Type='DateTime'><Today /></Value>
</Leq></Where></Query></View>" 
...

And finally, we simply change that code to the following to include OffsetDays (scroll to the right and look at the Today tag):

...
selectcommand="<View><Query>
<OrderBy><FieldRef Name='DiscussionLastUpdated' Ascending='TRUE'/></OrderBy>
<Where><Geq><FieldRef Name='DiscussionLastUpdated'/><Value Type='DateTime'><Today OffsetDays='-60'/></Value>
</Geq></Where></Query></View>" 
...

The list should now be filtered to only show discussions that have been updated within the past 60 days.  If you re-check the Filter settings for the list, it should look like so:

Reference blogs.msdn.com

In a previous post, I provided an outline for exporting models from Google Earth into 3DSMax.  The next step is to export the models from 3DSMax into UDK.

After a few weeks of extremely manual work, mostly revolving around textures, here is my neighborhood in UDK.

Notes:

1. Models exported from 3DS into UDK (originating from Google Earth) will not have collision data out of the box.  However, there is a simple work around:  In 3DS max, create a copy of every model, and rename all the copies with "UCX_" in front of the original name. Export the whole scene from 3ds max as an FBX.  When importing into UDK, in the Import Options menu, under Static Mesh -> Advanced, check "One Convex Hull Per UCX."  The collision with that isn't the greatest, but it's good enough to be able to walk around in the city for the most part and it's certainly a lot faster than doing all the collision by hand in 3ds Max. If you don't check the One Convex Hull Per UCX box (normally you wouldn't if you'd built the model and collision mesh yourself), the collision goes completely bonkers in UDK because of all the planes and concave meshes.
2. The textures associated to the models captured from 3D Ripper DX and imported into 3DS Max will be in the .dds format.  That's fine if you stay in 3DS Max.  However, if you wish to export the models with textures into UDK, the .dds extension is not recognized (thankfully, the texture coordinate data will still be there, just not the textures themselves).  To get around this, you need to batch convert the .dds files into .tga.  I used the IrFanView batch convert to accomplish this task.  However, you will also need to extend the extension capability of IrFanView by installing "All PlugIns."  This will allow the program to recognize the .dds extension, from there you can proceed with the batch conversions.  If you need assistance on the settings you should be using in IrFanView to create the tga files ready for UDK, reference this video (about 1:30min in).  Finally, with all of your tga textures ready to go, import them all into UDK.  Next, double click your static mesh in UDK to launch the Static Mesh Editor.  Under LODInfo > [0] > Elements is a listing of all the textures mapped to the model.  Simply expand out each element to reveal the texture mappings.  For each Material, locate and select the equivelant Material in the Content Browser, and click the green arrow next to the Material to "Use selected object in Content Browser."  This will overwrite the missing texture mapping with your imported tga texture.  Rinse and repeat until all Materials for the model have a tga association.

That was about as far as I decided to take this project in UDK.  Ultimately, the texture/mesh quality is just too low to be useful for anything other than extremely rough blocking out of a city.  To that end, it was a fun project.  🙂

Update: See my last post on this subject here to simplify a majority of the steps above.