This is just weird. I can easily load a 3.18 kernel and boot it from flash.
4.6.4 with your config won't work at all.
Could you put your kernel somewhere I can try it out? I'm starting to worry about my toolchain.
Also, do you use the bzimage or vmlinux? I would prefer to use the vmlinux but have no idea if that would work.
ron