'A'isha (Allah be pleased with her) reported:Allah's Messenger may peace be upon him) said to me: Had your people not been unbelievers in the recent past (had they not quite recently accepted Islam), I would have demolished the Ka'ba and would have rebuilt it on the foundation (laid) by Ibrahim; for when the Quraish had built the Ka'ba, they reduced its (area), and I would also have built (a door) in the rear.
حَدَّثَنَا يَحْيَى بْنُ يَحْيَى، أَخْبَرَنَا أَبُو مُعَاوِيَةَ، عَنْ هِشَامِ بْنِ عُرْوَةَ، عَنْ أَبِيهِ، عَنْ عَائِشَةَ، قَالَتْ قَالَ لِي رَسُولُ اللَّهِ صلى الله عليه وسلم " لَوْلاَ حَدَاثَةُ عَهْدِ قَوْمِكِ بِالْكُفْرِ لَنَقَضْتُ الْكَعْبَةَ وَلَجَعَلْتُهَا عَلَى أَسَاسِ إِبْرَاهِيمَ فَإِنَّ قُرَيْشًا حِينَ بَنَتِ الْبَيْتَ اسْتَقْصَرَتْ وَلَجَعَلْتُ لَهَا خَلْفًا " .
Muslim
The Book of Pilgrimage • Hadith 444