I have the exact same problem, and for years. It has not affected the integrity of the transmission, but I always wondered why and never got a straight answer. Some people think its a clog from a hardened seal that becomes partially unclogged when the seal warms up. It could be a maladjusted or sticking shift cable, or bad tranny fluid circulation when cold due to sludge or gunk build up in the syncro gears. It may also be a badly worn rear engine mount (some call a tranny mount). I've tried removing and cleaning the solenoids, flushing the tranny and replacing with Honda ATF, but these things have not worked for me. I still will attempt to fix the other nagging issues shortly....
It will happen when the engine generally idles at above 1100 RPMs or so. Once the idle comes down, the shift noise is much less apparent. I usually let the engine warm up for a minute or two, longer during the winter, less during the summer.