Saying there is a chance that it is NOT the CPU.... is not claiming that it isn't... it's simply less likely but like anything electronic, still a slim chance of it.
An HP computer from back in 2015-2016 ( the CPU was released late 2015 ) ... is going to make you rely on buying only used parts unless you are really lucky at finding "old new stock". Even the PSU you got to test with is probably used so can't be relied upon being good even if the seller stated it was in working order. At that time, the systems I built were using nothing less than a 450W and more than likely a 550w-650w PSU. The CPU you have is rated at only 35w, so the PSU isn't underpowered, but the system as a whole was built to the minimum spec for the time. HP was also using only partly industry compatible parts... so even replacing say a motherboard would give you trouble unless you also replace the case.
I always estimate a computer's lifespan VERY pessimistically. Using good above average parts... I give them a 7-10 year lifespan ( despite many of mine from after that long are still running ). A lot of an estimated lifespan is based on how good the electrical grid is in the area... and we have a lot of storms here... so being pessimistic is good so my clients aren't surprised if they do have a failure.
Considering the age of the system and current market price of a used equivalent... I wouldn't recommend fixing it unless someone just gave you the parts to test with. You would quickly reach or exceed it's value since it wasn't an expensive system when it was brand new. That isn't me laughing at it by any means... just stating what the approximate current value is what you'd potentially spend trying to fix it.
If the system is/was fast enough for most of your needs and you aren't needing a lot more.... I'd strongly consider getting a new MiniPC that will probably blow away the speeds you are seeing with your older system. Amazon is carrying a brand I trust named Minisforum and they have a lot of them to choose from. The one I'd mention is currently going for $380 ( may be higher in Hawaii due to shipping costs )... Amazon even has a $50 off coupon listed in their coupon section ( which isn't advertised, you have to search for the coupon area and then the brand name to find them ), how long they have a coupon is anyone's guess.
The one I'm looking at is a 6 core / 6 thread cpu running up to 4Ghz ... Your's is a 4c/4t at up to 2.8Ghz but using older tech so the IPC ( instructions per clock cycle ) are much lower. This one has 16 Gb of system memory and a 512Gb Nvme SSD. If you need a CD/DVD drive, you can get an external one for those times but most people are just using USB thumb drives now since they are faster and hold more data. You may also want a USB hub if you need more USB ports. The last thing you'd probably need is a HDMI to VGA adapter since I suspect you have an older monitor as well. The CD/DVD drives are probably in the $35-ish dollar range, the USB hubs vary a lot depending on needs... but a decent USB 3.0 4 port one will run no more than $25-30 and the video adapter can be found for under $10 all day long.
Here is a link to the one I just mentioned ... but remember they have different ones that list up to close to $1000 if you want to spend more ...
Amazon product
If you are considering something higher priced or even close to the upper end ones... I'd consider just doing a custom build instead which would run you close to $1100 instead and you'd end up with a lower end gaming video card as well. Of course, a custom built one will be a bit larger than what you have now... so space would still be a consideration.