Ok. After looking through this DIY and my old install photos, I understand what you're saying. My DIY says you have to remove only one side of the console exterior and from the sounds of it you chose the driver side. When I wrote this DIY(several months after actually doing the swap) I think I totally over looked the two wings on either side of the brackets. I think you may have to disassemble both the driver under panel and the passenger glove box. I had everything completely in pieces before I started putting everything back together:
I remember I had the brackets attached to the exterior console frame and I just angled it in and screwed everything together.
You're the first one to follow my DIY I think so sorry if you have to help me iron out a couple wrinkles.