Okay, so let's look at it.
"53 When Jesus had finished these parables, He departed from there. 54 He came to His hometown and began teaching them in their synagogue, so that they were astonished, and said, “Where did this man get this wisdom and these miraculous powers? 55 Is not this the carpenter’s son? Is not His mother called Mary, and His (friends), James and Joseph and Simon and Judas? 56 And (every day people), are they not all with us? Where then did this man get all these things?” 57 And they took offense at Him." (Mat. 13, NASB).
Changes the context, doesn't it? To understand the context the way you describe it, the verse would have to be read that way.