The Newcastle University study, published in the journal Diabetologia, analysed data from more than 400,000 pregnancies in North East England.
The risk of defects such as congenital heart disease and spina bifida were increased.
National guidelines already recommend having good control over blood sugar levels before trying to conceive.
Both Type 1 diabetes, which tends to appear in childhood, and Type 2 diabetes, largely as a result of diet, lead to problems controlling the amount of sugar in the blood.
This is known to cause problems in pregnancy, such as birth defects, miscarriage and the baby being overweight due to too much sugar.
There is concern that rising levels of diabetes, particularly Type 2, could make the issue worse.
Researchers analysed data from 401,149 pregnancies between 1996 and 2008 – 1,677 women had diabetes.
The risk of birth defects went from 19 in every 1,000 births for women without pre-existing diabetes to 72 in every 1,000 births for women with diabetes.
Their report said that sugar levels in the run-up to conception were the “most important” risk factor which could be controlled.
The lead researcher, Dr Ruth Bell from Newcastle University,”Many of these anomalies happen in the first four to six weeks.”
She said the number of pregnancies with poor sugar control were “more than we would like”.
“It is a problem when the pregnancy is not intended or when people are not aware they need to talk to their doctors before pregnancy,” she said.
Guidelines from the National Institute of Health and Clinical Excellence say women should reduce their blood sugar levels to below 6.1% before trying to have a baby.
Dr Bell said: “The good news is that, with expert help before and during pregnancy, most women with diabetes will have a healthy baby.
“The risk of problems can be reduced by taking extra care to have the best possible glucose control before becoming pregnant.”
The charity Diabetes UK funded the study. Its director of research, Dr Iain Frame, said: “We need to get the message out to women with diabetes that if they are considering becoming pregnant, then they should tell their diabetes healthcare team, who will make sure they are aware of planning and what next steps they should be taking.
“It also highlights the importance of using contraception if you are a woman with diabetes who is sexually active but not planning to become pregnant.”