Tasks can be deleted from multiple application screens.
From My Day:
1) From the Task List section, right click on the task.
2) Select Delete Task.
3) Are you sure? If yes, then click OK.
From Tasks:
1) To delete ALL completed tasks, select the Delete Completed button at the top of the page.
2) To delete a single task, right click and select Delete Task or click on the red "X" located accross from the task.
From Calendar:
1) Right click on the task and select Delete Task.