Fifth Circuit Affirms Penalty, Passport Revocation Case Dismissal

JAMES FRANKLIN,
Plaintiff-Appellant,
v.
UNITED STATES; CHARLES RETTIG, in his official capacity as Commissioner of Internal Revenue;
ANTONY BLINKEN, Secretary, U.S. Department of State;
JANET YELLEN, Secretary, U.S. Department of Treasury,
Defendants-Appellees.

United States Court of Appeals
for the Fifth Circuit

Appeal from the United States District Court
for the Northern District of Texas
USDC No. 3:20-CV-1303

Before KING, ELROD, and SOUTHWICK, Circuit Judges.^*

KING, Circuit Judge:

James Franklin appeals from the dismissal of his claims challenging tax penalties assessed against him, as well as the revocation of his passport pursuant to those penalties. He also appeals from the denial of an award of attorneys' fees under the Freedom of Information Act. For the following reasons, we AFFIRM.

I.

The Internal Revenue Service (IRS) found that James Franklin had failed to file accurate tax returns and had not reported a foreign trust of which he was the beneficial owner, and so it assessed penalties against him. Those penalties, assessed under 26 U.S.C. § 6677 on July 18, 2016, totaled $421,766. In 2018, the IRS began its collection efforts against Franklin, filing a federal tax lien and later levying on Franklin's Social Security benefits. The IRS also certified to the Department of State that Franklin had a “seriously delinquent tax debt” per 26 U.S.C. § 7345 (enacted under the Fixing America's Surface Transportation (“FAST”) Act), which led the State Department to revoke Franklin's passport.

In response to the penalties, Franklin, through counsel, filed a Freedom of Information Act (“FOIA”) request seeking: “(1) 'All relevant files and their contents . . . for the tax periods 1998 through 2017, including the entire administrative file relating to any auditor investigation' and (2) 'All relevant files, reports, letters, documents, or workpapers related to any penalty assessment under I.R.C. § 6677.'” In response, the IRS provided several documents from Franklin's administrative file and tax records; according to Franklin, those documents demonstrate that the IRS did not comply with statutory procedural requirements that must be satisfied before the assessment of penalties under § 6677.

Franklin did not file an administrative appeal of the IRS's response to his FOIA request; instead, he took the response at face value and assumed that the IRS had not complied with the procedural requirements (namely, that the penalties be approved in writing by a supervisor of the determining agent under 26 U.S.C. § 6751(b)). Franklin therefore filed an offer-in-compromise for a nominal sum, asserting a doubt of liability based on the purported procedural deficiencies. The IRS returned the offer without processing it, sending two separate letters that said respectively that (1) the IRS lacked jurisdiction to process the offer because “[i]t is regarding a foreign return and/or related issues,” and (2) “[o]ther investigations are pending that may affect the liability sought to be compromised or the grounds upon which it was submitted,”. Franklin then sent a second offer-in-compromise based on the same grounds and offering the same nominal settlement.

After these offers-in-compromise were unsuccessful, Franklin filed this suit. He asserted various claims related to the alleged procedural failure under § 6751(b) (collectively, the “§ 6751(b) Claims”); those claims were brought under 26 U.S.C. §§ 7345, 7432, and 7433; 28 U.S.C. § 2410; the Declaratory Judgment Act; and the Administrative Procedure Act. Franklin also challenged the constitutionality of the FAST Act's passport-revocation scheme, asserting that it violated his rights under the Fifth Amendment. Franklin later amended his complaint to seek attorneys' fees under FOIA after the Government, in its Motion for Partial Dismissal, attached exhibits that had not been produced in response to Franklin's FOIA request, which the Government asserted demonstrated that the penalties had been approved by the relevant supervisor. In addition to its assertions that the procedural requirements had been satisfied, the Government's motion also sought dismissal of the § 6751(b) Claims for lack of jurisdiction; the Government later updated its motion and sought either dismissal or summary judgment on Franklin's constitutional and FOIA claims.

The district court dismissed all of Franklin's claims. It first found that it lacked jurisdiction over each of the various § 6751(b) claims, finding that each was a prohibited collateral attack on the existence or validity of Franklin's tax liability for which the United States had not waived sovereign immunity. While the district court found that it did have jurisdiction over Franklin's Fifth Amendment challenge to the FAST Act's passport-revocation scheme, it also dismissed that claim after finding that the law was constitutional under rational-basis review.¹ Lastly, the district court found that while Franklin was eligible for attorneys' fees under FOIA (because his lawsuit prompted the IRS to release documents it alleged showed compliance with § 6751(b)'s procedural requirements), he was not entitled to an award of fees. Franklin timely appeals.

II.

We first consider whether the district court was correct to find that it lacked subject-matter jurisdiction over Franklin's various claims challenging the tax penalties. “We review questions of subject matter jurisdiction, including sovereign immunity determinations, de novo.” Daniel v. Univ. of Tex. Sw. Med. Ctr., 960 F.3d 253, 256 (5th Cir. 2020). When considering whether the United States has waived its default sovereign immunity, Franklin “bear[s] the burden of showing Congress's unequivocal waiver.” Freeman v. United States, 556 F.3d 326, 334 (5th Cir. 2009) (quoting St. Tammany Parish ex rel. Davis v. Fed. Emergency Mgmt. Agency, 556 F.3d 307, 315 (5th Cir. 2009)). His claims must be “brought in exact compliance with the terms of a statute under which the sovereign has consented to be sued.” Lewis v. Hunt, 492 F.3d 565, 571 (5th Cir. 2007) (quoting Hussain v. Bos. Old Colony Ins. Co., 311 F.3d 623, 629 (5th Cir. 2002)).

Franklin's panoply of claims share a common thread: he asserts he is entitled to damages for various IRS actions because they were based on penalty assessments that were invalid ab initio due to the IRS's failure to follow § 6751(b)'s procedural requirements. The district court was correct to find that each of these claims was a prohibited attempt to collaterally attack the actual penalties assessed. Under the Anti-Injunction Act, Congress has provided that, absent limited exceptions, “no suit for the purpose of restraining the assessment or collection of any tax shall be maintained in any court by any person.” 26 U.S.C. § 7421(a). Thus, when challenging the validity of a tax liability, the general rule is that a taxpayer must “pay first and litigate later.” Flora v. United States, 362 U.S. 145, 164 (1960). “[O]nce a tax has been assessed, [a] taxpayer . . . has no power to prevent the IRS from collecting it;” instead, the taxpayer must “pay the tax in full, and then sue for a refund.” Jones v. United States, 889 F.2d 1448, 1449-50 (5th Cir. 1989). Courts have zealously guarded this rule, recognizing the importance of the government's ability “to assess and collect taxes alleged to be due without judicial intervention” so that “the United States is [assured] of prompt collection of its lawful revenue.” Enochs v. Williams Packing & Navigation Co., 370 U.S. 1, 7 (1962).

We do not shirk that duty today. Each of Franklin's claims impliedly challenges the validity of the tax assessment itself. It does not matter that the procedural deficiency Franklin alleges occurred can technically be construed as occurring in a separate examination phase or can be characterized as occurring “pre-assessment.” The Supreme Court has clarified these discrete steps, see Direct Mktg. Ass'n v. Brohl, 575 U.S. 1, 9-10 (2015), and held that a challenge to reporting requirements backed by a tax penalty can proceed, CIC Servs., LLC v. IRS, 141 S. Ct. 1582, 1588-89 (2021). But it has reaffirmed that a challenge to the assessment or collection of a tax itself is still barred. See Id. at 1589.

Thus, the distinctions of the various phases of the tax process found in those cases do not aid Franklin here. This is so because Franklin's case is based on a cascade of inferences that necessarily includes a challenge to the assessment itself: proper procedures were not followed, therefore the IRS was not allowed to assess the penalties, therefore the assessed penalties were invalid, and therefore the IRS engaged in a cavalcade of actions based on the invalid penalties for which Franklin is owed damages. Franklin cannot state a claim without following that line of logic, as his challenges are all based on the alleged procedural deficiency that rendered the assessment void from the start. And that chain of reasoning features a defective link: it is based on an assumption that the assessment itself was void. Thus, his claims represent a challenge to the validity of the assessment over which the courts do not have jurisdiction.

A look at each of Franklin's claims confirms that conclusion. His § 7432 claim seeks damages for failure to release a lien — because the lien is based on penalties he states are invalid. His § 7433 claim seeks damages for wrongful collection activities — namely, trying to collect invalidly assessed penalties. His § 2410 claim seeks to quiet title to the property subject to the IRS's lien — because the lien stems from allegedly invalid penalties. He seeks a declaratory judgment and consideration under the Administrative Procedure Act — that the penalty assessments are invalid due to the procedural deficiencies. And, in addition to his constitutional challenge to the revocation of his passport, he seeks review of the certification that he has a “seriously delinquent tax debt” — because the tax debt is based on allegedly invalid penalties. At every turn, Franklin seeks to overturn the penalties, restrain collection of them, or otherwise cast doubt on the validity of the assessment. The government has not waived its sovereign immunity for those challenges, and so the district court was correct to dismiss them for lack of jurisdiction.

III.

We turn to Franklin's challenge to the constitutionality of the FAST Act's passport-revocation scheme. The district court dismissed that claim under Rule 12(b)(6). “We review a district court's ruling on a motion to dismiss de novo, 'accepting all well-pleaded facts as true and viewing those facts in the light most favorable to the plaintiffs.'” Anderson v. Valdez, 845 F.3d 580, 589 (5th Cir. 2016) (quoting Dorsey v. Portfolio Equities, Inc., 540 F.3d 333, 338 (5th Cir. 2008)).

Franklin's constitutional claim is based on his substantive due process rights under the Fifth Amendment. Substantive due process “protects individual liberty against 'certain government actions regardless of the fairness of the procedures used to implement them.'” Collins v. City of Harker Heights, 503 U.S. 115, 125 (1992) (quoting Daniels v. Williams, 474 U.S. 327, 331 (1986)).

Generally, liberty interests protected by the Fifth Amendment are considered under one of two standards: strict scrutiny and rational-basis review. If a right is fundamental, strict scrutiny applies. Washington v. Glucksberg, 521 U.S. 702, 720-21 (1997). To survive strict scrutiny, a governmental restriction of a fundamental right must be “narrowly tailored to serve a compelling state interest.” Id. at 721 (quoting Reno v. Flores, 507 U.S. 292, 302 (1993)). If a right is not fundamental, then rational review is applied, and the restriction at issue survives as long as it is “rationally related to a legitimate government interest.” Reyes v. N. Tex. Tollway Auth., 861 F.3d 558, 561 (5th Cir. 2017).

In turn, to decide if the implicated right is fundamental, that right must be “carefully describe[d].” Malagon de Fuentes v. Gonzales, 462 F.3d 498, 505 (5th Cir. 2006). In the instant case, the district court correctly defined the right as “the right to international travel.” We next consider whether that right is “'deeply rooted in this Nation's history and tradition' . . . and 'implicit in the concept of ordered liberty,' such that 'neither liberty nor justice would exist if [the right was] sacrificed.'” Glucksberg, 521 U.S. at 721 (first quoting Moore v. City of E. Cleveland, 431 U.S. 494, 503 (1977), then quoting Palko v. Connecticut, 302 U.S. 319, 325, 326 (1937)). The Supreme Court has cautioned that, because declaring a right as fundamental “to a great extent . . . place[s] the matter outside the arena of public debate and legislative action,” courts should “exercise the utmost care whenever [they] are asked to break new ground in this field.” Id. at 720 (quoting Collins, 503 U.S. at 125).

Given the Supreme Court's guidance on restrictions to international travel, we cannot find that it is a fundamental right such that restrictions on it merit strict scrutiny. It is true that three of the Supreme Court's cases hinted at the possibility that the right to international travel is fundamental. In Kent v. Dulles, the Court stated that “[t]he right to travel is a part of the 'liberty' of which the citizen cannot be deprived without the due process of law under the Fifth Amendment” and that “[f]reedom of movement is basic in our scheme of values.” 357 U.S. 116, 125-26 (1958). In Aptheker v. Secretary of State, the Court held that a law restricting the grant of passports to members of the Communist Party was an unconstitutional violation of the petitioner's rights under the Fifth Amendment. 378 U.S. 500, 504-05 (1964). In doing so, the Court stated that the law “swe[pt] too widely and too indiscriminately across the liberty guaranteed in the Fifth Amendment” and noted that “Congress ha[d] within its power 'less drastic' means of achieving the congressional objective of safeguarding our national security.” Id. at 512-14 (quoting Shelton v. Tucker, 364 U.S. 479, 488 (1960)). And in Zemel v. Rusk, the Court seemed to compare the right to international travel favorably to the fundamental right of interstate travel; it upheld a ban on travel to Cuba based on “the weightiest considerations of national security,” analogizing to the fact that the “freedom [of interstate travel] does not mean that areas ravaged by flood, fire or pestilence cannot be quarantined.” 381 U.S. 1, 15-16 (1965). The Court stated that just as interstate travel can sometimes be abridged to protect “the safety and welfare of the area or the Nation as a whole[,] [s]o it is with international travel.” Id.

However, later Supreme Court decisions have consistently and decisively refuted these earlier suggestions, going to great pains to separate the right to international travel from the fundamental right to interstate travel. In Califano v. Aznavorian, the Court noted that its past cases “often pointed out the crucial difference between the freedom to travel internationally and the right of interstate travel.” 439 U.S. 170, 176 (1978). Therefore, the Court made clear that regulations affecting international travel are “not to be judged by the same standard applied to laws that penalize the right of interstate travel.” Id. at 177. In so holding, the Court further explained that “[t]he constitutional right of interstate travel is virtually unqualified” while “the 'right' of international travel has been considered to be no more than an aspect of the 'liberty' protected by the Due Process Clause of the Fifth Amendment.” Id. at 176 (quoting Califano v. Torres, 435 U.S. 1, 4 n.6 (1978)). “As such this 'right,' the Court has held, can be regulated within the bounds of due process.” Id. (quoting Torres, 435 U.S. at 4 n.6).

The court reiterated this distinction when considering the revocation of the passport of a former CIA officer who threatened to reveal state secrets in Haig v. Agee, stating again that “the freedom to travel outside the United States must be distinguished from the right to travel within the United States.” 453 U.S. 280, 306 (1981). And it hammered home the point in Regan v. Wald, explicitly stating that any implication in Kent that the right to international travel was concomitant with the right to interstate travel had been “rejected in subsequent cases.” 468 U.S. 222, 241 n.25 (1984). The Supreme Court also noted that Kent and Aptheker concerned restrictions on travel based on a person's beliefs and associations; their discussions of the purported fundamental right to international travel thus hinged on protecting rights derived from the First Amendment that were not implicated in later cases (and that are not implicated here). See Haig, 453 U.S. at 304; Regan, 468 U.S. at 241 (“First Amendment rights . . . controlled in Kent and Aptheker.”). Taken together, these decisions of the Supreme Court make clear that the right (or, more aptly, freedom) to travel internationally is not fundamental, and thus that restrictions on international travel like the FAST Act's passport-revocation scheme are not to be judged under strict scrutiny.

We note that the above conclusion does not necessarily end the analysis. Franklin is correct in his assertion that the right to international travel is not a new creation unmoored from our past, but instead can be traced through the ages from Magna Carta to Blackstone to the Declaration of Independence to the modern Universal Declaration of Human Rights. Recognizing that fact, two esteemed members of our sister circuits have concluded that the right to international travel holds a place somewhere between the two poles that anchor our substantive-due-process jurisprudence and should be considered under intermediate scrutiny. Maehr v. U.S. Dep't of State, 5 F.4th 1100, 1115 (10th Cir. 2021) (Lucero, J., concurring in the judgment) (applying intermediate scrutiny to the FAST Act's passport-revocation scheme); Eunique v. Powell, 302 F.3d 971, 978 (9th Cir. 2002) (McKeown, J., concurring) (applying intermediate scrutiny to a law revoking passports of those seriously in arrears on child-support payments). Intermediate scrutiny requires that the challenged restriction “must serve important governmental objectives and must be substantially related to achievement of those objectives.” Craig v. Boren, 429 U.S. 190, 197 (1976).

However, we need not decide whether restrictions that effectively deprive an individual of the ability to travel outside our borders, like the passport-revocation scheme before us, should be judged under rational-basis review or intermediate scrutiny. Even under the higher standard of intermediate scrutiny, the FAST Act's passport-revocation scheme is constitutional. The government's interest in collecting taxes, which animates the FAST Act's passport-revocation scheme, is undoubtedly an important one. See, e.g., Hernandez v. Comm'r, 490 U.S. 680, 699 (1989) (“[E]ven a substantial burden would be justified by the 'broad public interest in maintaining a sound tax system'” (quoting United States v. Lee, 455 U.S. 252, 260 (1982))); Flora, 362 U.S. at 154 (“It is essential to the honor and orderly conduct of the government that its taxes should be promptly paid[.]”). The passport-revocation scheme is also clearly connected to that goal: delinquent taxpayers will be well-incentivized to pay the government what it is owed to secure return of their passports, and those same taxpayers will find it much more difficult to squirrel away assets in other countries if they are effectively not allowed to legally leave the country.

The FAST Act's passport-revocation scheme also does not sweep beyond what is necessary to achieve Congress's goal of trying to recoup the $5.8 billion or more in delinquent taxes owed to the government. The government is not authorized to seize the passport of any person who owes any taxes. Instead, the scheme is focused on those with serious tax debts and provides several procedural safeguards through both the tax process and, ultimately, through a cause of action should the certification itself be erroneous. Congress was within its rights to provide the IRS another arrow in its quiver to support its efforts to recoup seriously delinquent tax debts. And, importantly, what Congress provided was an arrow, not a bazooka. Its chosen tool is carefully aimed at the problem, not fired indiscriminately with grave risk of collateral damage to the rights of those not covered by the scheme. Under even intermediate scrutiny, the passport-revocation scheme is constitutional. See Eunique, 302 F.3d at 978 (McKeown, J., concurring) (upholding a similar passport-revocation scheme for those in serious arrears on child-support payments under intermediate scrutiny). The district court was correct to dismiss Franklin's challenge.

IV.

We last consider whether the district court correctly denied Franklin attorneys' fees related to his FOIA request. A district court's decision on whether to award attorneys' fees under FOIA is reviewed for abuse of discretion. Batton v. IRS, 718 F.3d 522, 525 (5th Cir. 2013). An award of fees under FOIA² is considered under a two-pronged test: first considering fee eligibility and second considering entitlement to the award of fees. Id. “The eligibility prong asks whether a plaintiff has substantially prevailed and thus may receive fees. If so, the court proceeds to the entitlement prong and considers a variety of factors to determine whether the plaintiff should receive fees.” Id. (quoting Brayton v. Office of the U.S. Trade Representative, 641 F.3d 521, 524 (D.C. Cir. 2011)). When considering entitlement to fees, courts look to four factors: “(1) the benefit to the public deriving from the case; (2) the commercial benefit to the complainant; (3) the nature of the complainant's interest in the records sought; and (4) whether the government's withholding of the records had a reasonable basis in law.” Id. at 527 (quoting Texas v. ICC, 935 F.2d 728, 730 (5th Cir. 1991)).

The district court did not abuse its discretion in declining to award fees. Franklin's lawsuit is far afield from the purposes for which FOIA, and its attorneys' fees provision, were designed. There is no public value in the information, and no value for anyone other than Franklin. Instead, Franklin only sought the information to aid him in his personal fight with the IRS regarding his tax penalties. When considering FOIA attorneys' fees, we have generally looked with disfavor on cases with no public benefit. This is especially true when the information seeker was motivated by a private commercial interest (such as reversing sizeable tax penalties to avoid paying hundreds of thousands of dollars to the government). See, e.g., ICC, 935 F.2d at 733 (“We are persuaded that the information ordered disclosed in Texas's suit is so devoid of public benefit that the trial court was within its discretion in denying Texas's request for fees.”); Blue v. Bureau of Prisons, 570 F.2d 529, 533-34 (5th Cir. 1978) (“Thus the factor of 'public benefit' does not particularly favor attorneys' fees where the award would merely subsidize a matter of private concern; this factor rather speaks for an award where the complainant's victory is likely to add to the fund of information that citizens may use in making vital political choices.”).

That is the case here. Our collective treasury of knowledge is made no richer through knowing whether or not an IRS supervisor signed forms authorizing the penalties assessed against Franklin. The public gains no benefit from that information; only Franklin could as he seeks to overturn those penalties. The district court did not abuse its discretion in deciding not to award him attorneys' fees.

V.

For the foregoing reasons, we AFFIRM.