It was narrated that Aishah said:"The Messenger of Allah said: "Were it not for the fact that your people have recently left disbelief, I would have knocked down the House and rebuilt it on the foundation of Ibrahim, peace be upon him, and I would have given it a back door. For when the Quraish built the House, they made it too small.'"
أَخْبَرَنَا إِسْحَاقُ بْنُ إِبْرَاهِيمَ، قَالَ أَنْبَأَنَا عَبْدَةُ، وَأَبُو مُعَاوِيَةَ قَالاَ حَدَّثَنَا هِشَامُ بْنُ عُرْوَةَ، عَنْ أَبِيهِ، عَنْ عَائِشَةَ، قَالَتْ قَالَ رَسُولُ اللَّهِ صلى الله عليه وسلم " لَوْلاَ حَدَاثَةُ عَهْدِ قَوْمِكِ بِالْكُفْرِ لَنَقَضْتُ الْبَيْتَ فَبَنَيْتُهُ عَلَى أَسَاسِ إِبْرَاهِيمَ عَلَيْهِ السَّلاَمُ وَجَعَلْتُ لَهُ خَلْفًا فَإِنَّ قُرَيْشًا لَمَّا بَنَتِ الْبَيْتَ اسْتَقْصَرَتْ " .
Nesai
The Book of Hajj • Hadith 2901