Dean Keates bemoaned his side's slow start as the Saddlers slipped to a second successive home defeat.
Billy Kee's seventh-minute penalty gave newly-promoted Accrington a 1-0 win, preserving their unbeaten away record.
Stanley keeper Connor Ripley kept the Saddlers at bay with a string of fine saves, on what was a frustrating afternoon.
“You can’t give a team a head start and we gave them a head start,” Keates said.
“We were not good enough. The keeper made some good saves but that is football.
“It was a little bit scrappy and we have spoken to the lads about that.
“We will clip it all together and look back at it on Monday and make sure it’s right for Tuesday [against Shrewsbury].
“We were just too bitty. The final product, the end ball, it just wasn’t good enough.
“When we did get through, their keeper has pulled off some good saves.”