When your off and on with a guy, when should you walk away?

Long story short, I've been off & on with this guy for about a little over a year, he's a good person but we are both in school in the same school doing the same thing so I understand the scheduling issues. Point is he is the one person I cannot figure out, I am very good at reading people, he always throws me for a loop. Sometimes a large amount of time will go by and I won't hear from him and he tells me how much he missed me. Let's say the actions do not correlate with what he says. He is just that type of guy that freaks out with commitment but he's not a player. I know it sounds like i'm defending him or justifying to my self but I say that because religion is very important to him and even his reputation before hand was known to be a serious guy. Point is when we are together it's amazing, I love it, but I just get so tired of waiting and wanting more from him. Part of me what's to tell him how I feel then I think what's the point? I'm not sure what to do, people that knows us mutually say he is a really good guy and just wait it's worth it, and other people say how long do you intend to wait. I'm not in rush to get married or anything I just want to spend more time with him. I just don't know how to approach this any ideas?