The objective of this research was to unravel the digestion of high amylose (AM) starch (HAS) granules through comparison of digestion of eight different types of HAS granules obtained from maize, potato, wheat, and barley. Unexpectedly, the resistant starch content (RS) of the HAS granules, ranging from 21% to 63%, did not correlate with the apparent AM content (AAC), which ranged from 34% to 97%. Instead, the pivotal factor governing digestibility was identified as the proportion of granules with smooth surfaces, in conjunction with the localized organization related to the arrangement of AM chains. Specifically, HAS granules originating from potato and maize, characterized by predominantly smooth-surfaced granules, as well as a higher double helix and B-type crystalline contents, exhibited higher RS content. Conversely, HAS granules sourced from wheat and barley, distinguished by a prevalence of granules with rough surfaces and high amorphous regions, displayed lower RS content. Furthermore, while potato and maize-derived HAS granules underwent minimal reorganization during digestion, those from wheat and barley underwent substantial molecular realignment. This phenomenon is likely attributed to the enhanced long AM molecules within wheat and barley, resulting in more significant degradation and molecular restructuring during digestion. The reorganized segments demonstrated increased resistance to enzymatic digestion. Thus, this study yields valuable insights into the mechanisms of the resistance of HAS granules to enzymatic digestion, emphasizing that AAC itself, within the range explored, does not emerge as a critical factor affecting their digestibility. The RS of HAS likely encompasses both pre-existing resistant structures and reorganized structures that form during digestion.