HP’s done it again. They’ve managed to break their custom ESX ISO on the G6 and G7 servers. I suspect it’s the same for G8.
If like me you found yourself perplexed by the PSOD (pink screen of death) after upgrading from ESX 6.0 to 6.5, keep on reading for the fix (skip to the bottom for the download link to an already fixed ISO).
It seems the “hpe-smx-provider” driver version 6.5.0 from the ESX 6.5 ISO is causing the PSOD.
The fix and the drawbacks
(Simply) Replace the driver with it’s older counterpart (version 6.0.0) and reinstall (see below for the procedure or check the “download link” section at the very end for an already fixed ISO). Unfortunately if you decide to create a custom ISO with the previous version of “hpe-smx-provider”, you will no longer be able to upgrade to any future ESX version and will need to do a full installation every time (thank you HP!). That being said, here is the procedure to customize the ISO:
Customize your own HPE ESX ISO
- Install PowerCLI from here. At the time of this writing, the latest version was 6.5.0R1.
- Download both the ESX 6.5 and 6.0 offline bundles and save them to a convenient place (ex: C:\HP) — download from this link. To make it easier, rename them to something like HPE_ESX6.5.zip and HPE_ESX6.0.zip.
- Open up PowerCLI and do the following:Add the 6.5 bundle
cd C:\HP Add-EsxSoftwareDepot -DepotUrl HPE_ESX6.5.zip
# check that the profile was loaded and the vendor is HPE
# clone the profile (feel free to specify whatever you like for the “vendor” when prompted)
New-EsxImageProfile -CloneProfile HPE-ESXi-6.5.0* -Name "HPE-ESX-6.5-CUST"
# double check the profile was indeed cloned
# Remove the “hpe-smx-provider” driver from the clone
Remove-EsxSoftwarePackage HPE-ESX-6.5-CUST hpe-smx-provider
# Add the 6.0 bundle
Add-EsxSoftwareDepot -DepotUrl HPE_ESX6.0.zip
# check the profile
# check driver versions from both bundles
Get-EsxSoftwarePackage | findstr smx
# Add hpe-smx-provider version 6.00 to the custom profile
add-esxsoftwarepackage -imageprofile HPE-ESX-6.5-CUST -softwarepackage "hpe-smx-provider 600.03.11.00.9-2768847"
# Export the profile to an ISO
Export-EsxImageProfile -ImageProfile HPE-ESX-6.5-CUST -ExportToIso -filepath "HPE-ESX-6.5-CUST.iso"
The following images have hpe-smx-provider version 600.03.11.00.9-2768847. Nothing else was changed. The 6.5.0d (build 5310538) is running on an HP Proliant DL380 G7 since March 2018 and was also tested on a DL380 G6 for a couple of months. For convenience, I’ve also built a 6.7.0 ISO and at the request of a reader, 6.7.0 Update1 from April of 2019:
41 thoughts on “ESX 6.5 and 6.7 on HPE G6/G7 server PSOD fix”
Thank you for the custom ISO
You are welcome.
thank you so much for your information and the custom ISO
i installed esxi custom iso 6.7 on the HP G8 successfully and without any error or crash, But i have a question, what about esxi update?
i was trying to update esxi 6.7 to the latest version with command “esxcli”, i got following error:
[dependencyerror] vib qlc_bootbank_qedrntv_220.127.116.11-1oem.618.104.22.16835516 requires qedentv_ver = x.9.18.0, but the requirement cannot be satisfied within the imageprofile. please refer to the log file for more details.
i was resolved the problem and updated esxi by this solution:
so, i wanted to ask you this solution is it ok or not?
The profile is a custom one, due to the “hpe-smx-provider” package downgrade and that is why you get that error message. Because of that, no upgrades can be made in the traditional way. Your only option is to create a custom ISO every time you want to upgrade. HP really screwed us over this time around.
Any Chance to have a customized ISO for the latest May2019 Release of ESX 6.7 ?
I will create one in the following days and reply again to your comment.
I could only find the image from April of 2019. As such, here it is: https://drive.google.com/open?id=13n9rJeL0QGrTvJiQzEA_muMsHwJ_9dvt
Please beware, it’s untested since I have no hardware at the moment where I can run tests.
I have a question; i updated to the April version Custom ISO (6.7), but i do not have the HTML5 update of the web client.
Why is that?
It’s probably not included in the original ISO. The only change the custom image has is the 1x driver.
Do you know if it is included in one of the other images?
I know that the original 6.5 has the HTML 5 web client, right?
No idea if it’s included on the other images or not since I can’t run 6.7 on my DL380 G7 due to unsupported CPU.
Yes, the HTML 5 client is included in the 6.5 image.
Thanks for preparing the ISO. I’ve tried to update from 6.5 to 6.7U1 on a HP DL 380 G7 yesterday. It worked in the end but I ran into some problems. At first i tried to update, so I mounted the ISO as a virtual DVD-Drive using ILO and was able to boot from it. The install aborted at first telling me there was a caching issue. I made sure settings for Swap and Scratch were set properly to the local datastore (ESXI itself runs from a SD-Card) and tried again. This time the installation went up to 78% but then stopped with a generic failure I don’t even remember. At first I didn’t really worry about it, I had a configuration backup (created using the “vim-cmd hostsvc/firmware/backup_config” command), so I did a clean install and tried to restore using the “vim-cmd hostsvc/firmware/restore_config” command.
This also failed and finally made me worry. A common error seems to be a different UUID which wasn’t the case as this was the same machine. I tried a couple of times and finally gave up.
In the end I found out (related blogpost: https://768kb.wordpress.com/2014/06/26/esxi-config-backup-restore/), actually its not so hard to do a manual restore without any ESXi cli-tool, Power CLI or vSphere functionality. It’s enough, at least it was for me, to extract the local.tgz file from the state.tgz in the backup file to the root of your ESXI host. After I did this and replaced the files with the ones from the backup my host came back up with the old configuration and everything worked again. In the end this took me about 3 hrs, far from the 30 mins I hoped for…
Any Chance to have a customized ISO for the HP ESXi 6.7 U2 ?
Any Chance to have a customized ISO for VMware-ESXi-6.7.0-Update2-13006603-HPE-Gen9plus-670.U22.214.171.124.8-Apr2019.iso ?
thanks in advance
I’m currently unable to download the 6.7U2 offline bundle from the official VMWare website. I’ll give it a try later on and let you know.
PS: You can also customize the ISO yourself by following the tutorial.
I’m trying to do this for the latest ESXi 6.7 U2 (also from April 2019) from the HPE link you provide and it doesn’t find hpe-smx-provider:
WARNING: Package hpe-smx-provider could not be found in image profile HPE-ESX-6.7-CUST, not removing.
But if I execute Get-EsxSoftwarePackage | findstr smx it finds “smx-provider”:
smx-provider 670.03.15.00.3-7535516 HPE 08/10/2018 11…
Is it a rename or this version already is fixed? What to do?
If you downloaded the ISO from the official website, it might be that HP renamed the package. In that case, simply replace that with the 6.0 version.
Can I upgrade esxi 6.0 to 6.7 using VCSA Upgrad, or have to be trough cli and mounted image??
You cannot do upgrades, you need to do a clean install.
I tried your “VMware-ESXi-6.7.0-Update1-11675023-HPE-Gen9plus-670.U126.96.36.199.19-Apr2019_CUSTOM.iso” in HP Proliant DL360 G7. It was pretty OK, but some of the system sensors (mainly System Chassis 1 UID Ligh and System Chassis 2 Sys. Health LED) were unknown. Then the whole Storage hw monitoring (including HPE Smart Array P410i) seemed to be unknown. Maybe some more packages than just hpe-smx-provider should be replaced? Or is that just mission impossible to do 100% esxi 6.7U1 for G7?
i’m trying to get the 10GB nic to work with esxi 6.0 or 6.5 ot 6.7
does this custom iso fix it the 10gb nic?
https://www.vmware.com/resources/compatibility/search.php Make sure your card is supported
successfully installed 6.7 update 3 hpe image without any modification on ml350 g6.
compared with 6.7: lost some hardware sensors (fans, power meter) and there is a warning alert about “system board 10 memory”. downgrading smx didn’t solve issues.
Hie, can you do the same for ESXI 7.0, I have an HP Proliant G6 Server working fine with your ESXI 6.7 image running brilliantly and looking forward to test 7.0 soon. Thanks in advance.
I’ll give it a try but it will take a while.
Wondering if you have this for 7.0 yet? Your other images have work very well for me! Thanks!
I do not
Hi any update for ESXi 7 for 360/380 G6 Servers?
I no longer have the option to test 🙁
Nice work, I have a DL380 G7 and when I try to boot either of these three ISOs, I still seem to get the following PSOD. https://photos.app.goo.gl/FUpY2bk6kkRn1nqaA
Any guidance would be helpful; not sure what to try next.
Not sure, I’d check the hardware first.
did you manage to solve this problem? i faced the same error and nothing helps.
Install iLO version 1.88 and BIOS 2018.02.22.
With newer versions I was getting the same error
Recently came across your blog, I’m going to take a stab at building/testing a custom image from your tutorial for a fresh install to 7.0U1 on my DL360 G8. I applaud you for being very accommodating to your commenters and building custom images. I am currently running 6.7U3 upgraded via CLI from the original 6.5U3 HPE image. It has been stable for .
If I want to install esxi 6.5 update 3 on G6. I should download esxi 6.5 u3 and esxi 6 and change as you mentioned on esxi 6.5 u3 iso? Is that right? I think the lastest esxi version is base for any hp servers. Right?
I have a hp dl980 g7 and hp ml350p g8 which is installed ESXi 6.0 version and working properly with no concern now.
I need to upgrade ESXi 6.0 to 6.5u3 and I created a custom iso which is mentioned above and because of the PSOD issue I didn’t upgrade the ESXi yet, and I made a customized iso and changed the hpe-smx-provider 6.5 with 6.0.
So if my host working properly with ESXi 6.0 now (did not upgrade to 6.5 because of PSOD known issue), can I upgrade esxi 6.0 to 6.5 with customed iso or do I have to fresh install anyway?
IIRC you need a fresh install
Trying to make a custom bundle for 6.7U3:
PowerCLI C:\users\ciprr008\Downloads> Remove-EsxSoftwarePackage HPE-ESX.6.7-CUST hpe-smx-provider
WARNING: Package hpe-smx-provider could not be found in image profile HPE-ESX.6.7-CUST, not removing.
So is the hpe-smx-provider no longer there?
I guess not.
for me it works but i got an error on this step too.
In my case it wasd a “copy and paste ” Problem :D.
When i paste the code in the cli shell it removes a Space Key betwen “Remove-EsxSoftwarePackage HPE-ESX-6.7-CUSThpe-smx-provider” it must be “Remove-EsxSoftwarePackage HPE-ESX-6.7-CUS Thpe-smx-provider” afrer setting the “space” Key it works for me.
At the end i got another error on creating the iso File “Export-EsxImageProfile : Could not find a trusted signer.” and to fix that i addet the Parameter “-NoSignatureCheck” at the end. And now i have a 6.7 U3 ISO
But haven not tested the installation at that moment…
Thanks for this Article it helps me a lot!