I don't have a huge amount to offer, as I say I tend not to disclose anything these days but I do recall something that Jervis Johnson said a while ago.
Apparently when Gav Thorpe wrote the last CSM Dex he was given very strict instructions on what he could and couldn't do. He was pretty much forced to streamline the Codex as the dev team thought the previous one was too complicated.
Gav really didn't have much say in the Dex, and wasn't happy with just how stripped down it was, but those were the wishes of the design department.
However Jervis said that as soon as the Dex was released they realised they had made a mistake, and started planning the new Dex almost immediately.
Suffice it to say this new Dex has been in planning for quite some time, and the idea was to reintegrate some of the elements from the Codex before last.
Jervis also said that the plan was to release the Legions book, but keep the current CSM Codex as a Renegades Codex.
This may be where the confusion of 2 books arises from, as far as I know the new Legions Dex will be in addition to the CSM one, not a replacement.
However that info is well over a year old now so once again, don't shoot the messenger.
I believe Jervis touched upon the point in one of his Standard Bearer articles, I will see if I can dig out which one, but it was ages ago. He then expanded on the article to somebody that I know, but as I say this was over a year ago now if memory serves.
In terms of the differences, the new Chaos Legions Codex is supposed to represent the Traitor Marines who fled to the Eye of Terror after the Heresy.
They are therfore Chaos Legionnaires rather than Chaos Marines, and are therefore different to the Marines who have since turned their backs on the Emperor and have gone renegade.
They are much older, and closer to their original Legions (World Eaters, Thousand Sons etc) and will have distinct rules to represent this.
In terms of Traitor Guard apparently the idea for a full Traitor Guard Codex was put forward but was rejected.
Rumours say that Traitor Guard will be available as a unit choice in the new Legions Codex (by taking Alpha Legion) but won't be available as a full army.
AFAIK the only current plans are for the Legions Codex, nothing else. The current CSM Codex will remain to represent Traitor Marines rather than Legionnaires.
CSM Codex stays viable for Renegade Marines.
Legions Codex will come to cover the older Legions like Death Guard and Thousand Sons INA DDITION to the old CSM Codex.
Traitor Guard idea considered but abandonned, now only a unit for Alpha Guard in Legions Codex.
Either way, time will tell which of us were correct, but most really has to do with SilverStraight posted, which is "things change" within the process of GW books/codices. Nice to see he also confirmed one of my earlier rumors, with his own "rumor" that it's already been mentioned that the current Chaos Codex will be referred to as "Chaos Renegades" with a White Dwarf update. As for the NEW Chaos Codex, a simple "Chaos: Legions" name should suffice.
Some of my previous info may actually be Apocalypse Formations stuff, that will be released around the same time as the new Codex. Maybe the Brazen Knights/Brass Knights/Blood Knights?