Years ago I was a corruption and then mafia prosecutor over a 10-year span—first for New York state, and later the U.S. Justice Department. Top tier criminal investigations always. I spent more time in investigative grand jury rooms than anywhere else, examining witnesses—crooked police, politicians and mobsters. Witness after witness was hostile to those probes, but no lawyer for any witness ever intruded into the sanctity of the grand jury chamber. The law simply didn't allow it (and from my vantage point, rightly so).

To me, lawyers for witnesses, typically, were simply in the way. They were often, to my mind, "obstructionists" and sometimes even outright suborners. I and my colleagues were invariably frustrated by the constant battle against witness contempt, dissembling, evasiveness and sometimes even outright perjury. These roadblocked our pathway to piercing the omnipresent blue wall of silence, or code of omerta. You know—truth, justice and the American way.

We were self-absorbed enough to believe that witnesses were not (entirely) forthcoming. After all, we, alone knew the truth! Inside the grand jury room, we prosecutors were the judge, jury and sometimes even the executioner (when, as in the Coliseum of old, we subliminally gave a thumbs down when asking the seated jurors to vote). And we were unimpaired by a lawyer for anyone observing the joust. Yes, with the witness alone inside the sanctum—no judge present, to be sure—we might more than sometimes throw hardballs, curve balls and sometimes even spitters designed to extract the truth (you know, the truth that we alone knew).

The grand jury system hasn't changed much from those days, except in one small way. Now, based on a reform of the late '70s, in a very rare circumstance New York law (but not federal) allows a witness to have his lawyer present, that is if he is required to waive immunity before testifying. Indeed, such a request nowadays by a prosecutor is rare indeed—typically only in a very close case where the prosecutor has genuine misgivings about indicting. For example, in the instance of a police shooting that resulted in death, where the officer hopes to talk the grand jury out of indicting and a sympathetic prosecutor encourages him to tell his side of the story to accomplish a "no true" bill.

It is so rare, that those colleagues at the criminal bar whom I unscientifically polled also haven't been inside a grand jury room since they left a prosecutor's office. I, myself, have never represented a witness or defendant in the grand jury room. That is, until now—one morning recently. My client who wasn't hostile in any way to the prosecutor's case, had waived immunity, and so I use the experience to share my observations.

My client was nervous, as would be any witness facing a prosecutor and 23 people staring at him dead on. He later told me, even though under New York law I had no right to object to questions or pretty much do anything while seated nearby (except to privately consult wit him, if necessary), that my mere presence gave him comfort that if he strayed in his testimony I would "be there" to help correct him.

But, here's the real thing. The prosecutor was utterly fair both to the witness and the targets of the investigation in his questioning. Would this prosecutor have been equally fair had I not been there? Probably so. But didn't my mere presence insist on that fairness, even if I never had reason to say a word or even grimace in his direction? When an outside observer or member of the bar is present (even if lined up on the side of the witness), isn't a prosecutor with an "intent"—prosecutors typically present cases to the grand jury with an unadorned "intent" to indict—more likely to toe the line when plying his trade?

I don't suggest here that a subject or target of a grand jury investigation should have a right to have his lawyer present. Too many potential problems with that. Grand juries are empaneled for investigations, not trials. There are obvious security risks in having defense attorneys present that might increase the possibility of obstruction by wayward clients who consequently learn who the witnesses are and what they're saying. And often, the prosecutor won't even know until the investigation's end who the target is.

But, why shouldn't someone, even if not the lawyer for the witness, be present in the grand jury room—if only as a silent policeman to chill a prosecutor willing to leave the straight and narrow? In some bizarre way, even if there was no cause for it that morning, I was the silent policeman. Meaning, if the prosecutor threw a spitball in my presence—even if not thrown at my client—a witness to it was there, even if not an umpire. A knowledgeable "witness" of a different sort.

There are too many instances of prosecutorial abuse in recent years in grand jury rooms—both state and federal. Maybe it's time to take a closer look at additional reforms to ensure that defendants aren't victimized by prosecutors on a wayward mission—or witnesses being pushed by the prosecutor improperly in the direction of the prosecutor's subjective view of the truth. Indeed, one might wonder how many wrongful convictions—say, in one witness identification or circumstantial evidence cases—have resulted from grand jury abuses resulting in indictments that led inexorably to convictions too.

And if one were to argue that a judge will ultimately review the grand jury testimony to ensure fairness in the grand jury presentation, a sterile grand jury transcript doesn't always tell the whole story, not to mention that a defendant will have suffered an indictment that perhaps wouldn't have been brought but for the prosecutor's misconduct. Further, lest it go unsaid, federal prosecutors who can actually indict simply on the basis of hearsay testimony, virtually never have to face a judge's inspection of a grand jury record.

I must admit that there were some moments in my grand jury days that, in retrospect, I'm not totally proud of. Had someone been there—a lawyer, or neutral magistrate of sorts—watching me every time I presented, things might sometimes have been a little different.

Hindsight sometimes yields better vision for the observer of the present.

Joel Cohen, a former New York state and federal prosecutor, is senior counsel at Stroock & Stroock & Lavan, and an adjunct professor at Fordham Law School. He is the author of the newly published I Swear: The Meaning of an Oath (Vandeplas Publishing, 2019)