The neighbouring countries have in recent years been at loggerheads over Ankara's cross-border military operations against the outlawed Kurdistan Workers Party (PKK) militants based in northern Iraq's mountainous region.
Iraq has said the operations violate its sovereignty, but Ankara says they are needed to protect itself.
Ties have improved since last year, when they agreed to hold high-level talks on security matters, and after a visit in April by Turkish President Tayyip Erdogan to Baghdad, where he said relations had entered a new phase.
Ankara and Baghdad held a fourth round of meetings this week as part of the dialogue mechanism. In March, Iraq labelled the PKK a "banned organisation in Iraq" - a move welcomed by Turkey.
Fidan, speaking alongside his Iraqi counterpart Fuad Hussein following their talks in Ankara, said the accord signed by the defence ministers of either side carried "historic importance", while Hussein said it was "the first in the history of Iraq and Turkey" in this field.
"Through the joint coordination and training centres planned in this agreement, we believe we can take our cooperation to the next level," Fidan said.
"We want to advance the understanding we are developing with Iraq on counter-terrorism through concrete steps on the ground," he added.
A Turkish diplomatic source said that, with the agreement, a Joint Security Coordination Centre would be established in Baghdad along with a Joint Training and Cooperation Centre in Bashiqa. Hussein, speaking about the Bashiqa training camp, said "the onus will lie on the Iraqi armed forces", without elaborating.
On Monday, Turkish Defence Minister Yasar Guler told Reuters that recent steps taken by Turkey and Iraq on counter-terrorism marked a turning point in ties, adding Ankara wanted Baghdad to go a step further and label the PKK a terrorist organisation as soon as possible.
The PKK, which has been waging an insurgency against the Turkish state since 1984, is designated a terrorist organisation by Turkey, the United States and the European Union. More than 40,000 people have been killed in the conflict.