The effects of overwatering, especially on succulents, can be seen for some time after you correct the issue. In fact the soil may not have even dried out enough yet, particularly if the plant happens to be overpotted. Unless you are in a very warm climate you probably would only need to water once per week at most, and two weeks might be better.
Can you tell us your growing conditions (indoors or outdoors? if outside where do you live?) and maybe even take a picture of the plant and pot?