Can I Lose Weight Eat Too Few Calories?

I feel like I need to make my own special little question about this because I can’t find similar questions asked by anyone as small as me. I feel like an idiot because I actually manage a health and wellness blog but I need answers.
I’ve been trying to lose weight since I had my daughter, over 18 months ago! I finally started monitoring my calories and, as it turns out (unless we go out to Chinese) I eat less than 1500 calories per day. Much less. Today I’ve had 1100 calories but my RMR is 1300. I burned over 600 calories with my workout today…but I kind of feel like working out again, better than anything that’s on TV tonight.
I am 164lbs and 4″11 inches tall. To be technically healthy, I need to be like 115 or less.
My question is this: do I need more calories in my diet? What am I doing wrong that I can’t lose weight? I do drink water, eat whole grains and veggies and blah blah blah, but there must be some information that I’m missing. I gained a pound per day when I tried to eat 2000 calories per day, even while working out. I don’t really understand….