This is just weird. I can easily load a 3.18 kernel and boot it from flash. 

4.6.4 with your config won't work at all.

Could you put your kernel somewhere I can try it out? I'm starting to worry about my toolchain.

Also, do you use the bzimage or vmlinux? I would prefer to use the vmlinux but have no idea if that would work.

ron