The main reason is so Jingles couldn't pull it out. However, with most stabbing wounds where the object is left in the body, EMTs will NOT pull the object out, which may cause greater blood loss. ER doctors or surgeons will remove the object in a controlled environment, IE, a trauma room or operating room, to reduce blood loss and repair the injury. The way most of these wounds are depicted in movies, the weapon is often pulled out immediately (for dramatic effect), however, it doesn't hold up in real life.
You could also argue that without the handle, the guards couldn't get any of Jimmy's fingerprints from it, implicating him immediately in the crime, hence, Jimmy drops it down the drain where they'd have less of a chance of finding it.
