Opinion polls suggest Kevin Rudd, who returned as prime minister three months ago, is trailing his opponent.
The rivals are split on how to tackle Australia's budget deficit, but both have pledged tough action to stop asylum seekers arriving by boat.
All citizens over 18 must vote by law.
More than 14 million people are expected to vote in Saturday's election. Shortly before the polls opened, Australia's election commission revealed that a record 3.2 million ballots had already been cast in early voting.
Mr Rudd called the election after ousting Julia Gillard in a leadership challenge in June, amid dismal polling figures. Ms Gillard had herself ousted Mr Rudd as prime minister in 2010.
Labor initially saw its figures improve significantly. But in recent weeks Mr Abbott has again broadened the gap. He has enjoyed the strident support of Rupert Murdoch's newspapers, and remains ahead in the opinion polls.
Polling stations in eastern Australia opened at 8:00 local time (22:00 GMT) and are set to close at 18:00. Voting in western states will end two hours later because of time zone differences.
Mr Abbott, accompanied by his wife and daughters, cast his ballot in his northern Sydney seat of Warringah shortly after polling began.
He then toured the city attempting to drum up support and was heckled by protesters at a polling station in the suburb of Arncliffe.
"Anything can happen today," he told reporters. "I don't believe the polls, Kevin Rudd doesn't believe the polls. I think it's still very close."
Mr Rudd, who appeared on breakfast TV in his home city of Brisbane, was otherwise less visible in public during the morning. He later joined his wife in voting in his Griffith seat.
"I'm very confident in people's judgement because they will assess what is best for our country's future, their community's future and their family's future," he told Channel 7 TV.
The two leading candidates made their final push on the campaign trail on Friday, with Mr Rudd appealing to undecided voters, saying they could close the gap.
Mr Rudd was campaigning in the New South Wales Central Coast, while Mr Abbott spoke at a guitar factory in Melbourne.
The prime minister emphasised the Labor government's economic record and said his priority was "jobs, more jobs and jobs, health, hospitals and broadband, and to keep support for cost of living pressures".
Mr Abbott said the coalition would "end the waste, stop the boats, and build roads of the 21st Century".
He also warned voters against "another hung parliament, and a weak and divided Labor-Green government".
"[The] only way to have a new way is to choose a new government," he said.
The economy has been a central issue to both campaigns. The next leader will be charged with managing the transition as the mining and resources boom subsides, amid slowing demand from China and slumping commodity prices.
Both parties have announced tough asylum policies to stem the number of people reaching Australia's shores by boat, an apparent cause of anxiety in some crucial swinging electorates.
Under a Labor plan, asylum seekers arriving by boat will be sent to Papua New Guinea for processing, and resettled there if found to be refugees.
Mr Abbott, meanwhile, says he will appoint a military commander to lead operations tackling people smugglers, and that asylum seekers granted refugee status would be limited to temporary renewable visas.
Both policies have been criticised by refugee rights groups. The UN has described Mr Rudd's policy as "troubling", while Greens Senator Sarah Hanson-Young described the coalition's policy as "cruelty and punishment for the sake of Tony Abbott looking tough".
Mr Abbott has also promised to repeal the government's unpopular carbon tax - a policy which has marked Australia out as a world leader on climate change legislation in the past three years.
Analysts say that voters are ready to reject the minority government and its carbon pricing, mining tax, and fibre-optic broadband network, in favour of smaller government, reduced debt, and a new domestic focus.