Ibn Umar was informed about the statement of Aisha that a part of al-Hijr is included in the magnitude of the Ka'bah. Ibn Umar said:By Allah, I think that she must have heard it from the Messenger of Allah (ﷺ). I think that the Messenger of Allah (ﷺ) had not given up touching both of them but for the reason that they were not on the foundation of the House (the Ka'bah), nor did the people circumambulate (the House) beyond al-Hijr for this reason.
حَدَّثَنَا مَخْلَدُ بْنُ خَالِدٍ، حَدَّثَنَا عَبْدُ الرَّزَّاقِ، أَخْبَرَنَا مَعْمَرٌ، عَنِ الزُّهْرِيِّ، عَنْ سَالِمٍ، عَنِ ابْنِ عُمَرَ، أَنَّهُ أُخْبِرَ بِقَوْلِ، عَائِشَةَ رضى الله عنها إِنَّ الْحَجَرَ بَعْضُهُ مِنَ الْبَيْتِ . فَقَالَ ابْنُ عُمَرَ وَاللَّهِ إِنِّي لأَظُنُّ عَائِشَةَ إِنْ كَانَتْ سَمِعَتْ هَذَا مِنْ رَسُولِ اللَّهِ صلى الله عليه وسلم إِنِّي لأَظُنُّ رَسُولَ اللَّهِ صلى الله عليه وسلم لَمْ يَتْرُكِ اسْتِلاَمَهُمَا إِلاَّ أَنَّهُمَا لَيْسَا عَلَى قَوَاعِدِ الْبَيْتِ وَلاَ طَافَ النَّاسُ وَرَاءَ الْحِجْرِ إِلاَّ لِذَلِكَ .
AbuDaud
The Rites of Hajj (Kitab Al-Manasik Wa'l-Hajj) • Hadith 1875