I get that the jean shorts are Smith's shtick, but he's starting to feel like a prisoner of his carefully cultivated slacker image. Do you think he is ever about to put on pants, then he blanches, thinking, "Can't let anyone see me without my jorts!" That is not healthy, Mr. Smith.
Live a full life! We already "get" you -- there's no need to keep it up! Could there be anything worse than having a drawer (potentially multiple drawers) that are just jorts? Save yourself the ignominy and explore other sartorial options for the good of mankind, or at least look into bringing back jams or something. At that point, we'd be willing to scratch off Jersey Girl.