memtest in many ways is not my favorite program. I would rather have a simple non-"GUI" tester, because in the past I've had trouble with it (it can fail) when it did not like the device it was trying to do its "GUI" on.
Also, the way memtest finds top of memory can fail on some chipsets. Have not seen that for a while but I did see it.
No idea if any of this is your problem but ...
ron