Arun Saw said to worry about his party