Aiming at the quality problems of ultrasonic flaw detection in the area near inner hole during the upsetting and punching process of free forging for large-scale 5083 aluminum-magnesium alloy flange, a finite element analysis model for the punching billet after upsetting was established, and the punching process was simulated to obtain the flow law of the material in the area near inner hole of flange billet. Then, it is concluded that the ultrasonic flaw detection quality defect in the area near inner hole of flange billet was caused by the radial shunt of the material flow in the area near inner hole during the reverse flattening process. Furthermore, based on the above analysis, it was proposed to adjust the reverse flattening process in the current process flow of forward punching,reverse flattening and trimming, that is, to keep the punch of forward punching in the reverse flattening process to constrain the inflow of material in the area near inner hole. The further simulation results show that the improved punching process can significantly improve the flow state of material in the area near inner hole and avoid the unfavorable factors of radial shunt that lead to the quality defects in the area near inner hole. The actual production shows that the adjusted process scheme effectively solves the problems of defects in the area near inner hole during the upsetting and punching process of large-scale flange.