Skip to content

Static Routes with nexthop non-functional for private gateways#12859

Open
bhouse-nexthop wants to merge 2 commits intoapache:4.22from
bhouse-nexthop:fix-static-route-pbr
Open

Static Routes with nexthop non-functional for private gateways#12859
bhouse-nexthop wants to merge 2 commits intoapache:4.22from
bhouse-nexthop:fix-static-route-pbr

Conversation

@bhouse-nexthop
Copy link

@bhouse-nexthop bhouse-nexthop commented Mar 18, 2026

Description

Static routes were only being added to the main routing table, but policy-based routing (PBR) is active on VPC routers. This caused traffic coming in from specific interfaces to not find the static routes, as they use interface-specific routing tables (Table_ethX).

This fix:

  • Adds a helper method to find which interface a gateway belongs to by matching the gateway IP against configured interface subnets
  • Modifies route add/delete operations to update both the main table and the appropriate interface-specific PBR table
  • Inserts iptables FORWARD rules for static routes that don't have a nexthop of a private gateway.
  • Uses existing CsAddress databag metadata to avoid OS queries
  • Handles both add and revoke operations for proper cleanup
  • Adds comprehensive logging for troubleshooting

Fixes #12857

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

See #12857 for the tested scenario

How did you try to break this feature and the system with this change?

@boring-cyborg
Copy link

boring-cyborg bot commented Mar 18, 2026

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

@bhouse-nexthop bhouse-nexthop changed the title Fix static routes to be added to PBR tables in VPC routers Static Routes with nexthop non-functional for private gateways Mar 18, 2026
@bhouse-nexthop bhouse-nexthop force-pushed the fix-static-route-pbr branch 2 times, most recently from f51b1ab to 1141a7a Compare March 18, 2026 18:35
@bhouse-nexthop bhouse-nexthop marked this pull request as ready for review March 18, 2026 19:03
@bhouse-nexthop bhouse-nexthop changed the base branch from main to 4.22 March 18, 2026 19:13
@bhouse-nexthop bhouse-nexthop changed the base branch from 4.22 to main March 18, 2026 19:14
Static routes were only being added to the main routing table, but
policy-based routing (PBR) is active on VPC routers. This caused
traffic coming in from specific interfaces to not find the static
routes, as they use interface-specific routing tables (Table_ethX).

This fix:
- Adds a helper method to find which interface a gateway belongs to
  by matching the gateway IP against configured interface subnets
- Modifies route add/delete operations to update both the main table
  and the appropriate interface-specific PBR table
- Uses existing CsAddress databag metadata to avoid OS queries
- Handles both add and revoke operations for proper cleanup
- Adds comprehensive logging for troubleshooting

Fixes apache#12857
@bhouse-nexthop
Copy link
Author

@weizhouapache can you approve the workflow runs and review this. I've just confirmed this fixes both of my issues.

When static routes use nexthop (gateway) instead of referencing a
private gateway's public IP, the iptables FORWARD rules were not
being generated. This caused traffic to be dropped by ACLs.

This fix:
- Adds a shared helper CsHelper.find_device_for_gateway() to determine
  which interface a gateway belongs to by checking subnet membership
- Updates CsStaticRoutes to use the shared helper instead of duplicating
  the device-finding logic
- Modifies CsAddress firewall rule generation to handle both old-style
  (ip_address-based) and new-style (nexthop-based) static routes
- Generates the required FORWARD and PREROUTING rules for nexthop routes:
  * -A PREROUTING -s <network> ! -d <interface_ip>/32 -i <dev> -j ACL_OUTBOUND_<dev>
  * -A FORWARD -d <network> -o <dev> -j ACL_INBOUND_<dev>
  * -A FORWARD -d <network> -o <dev> -m state --state RELATED,ESTABLISHED -j ACCEPT

Fixes the second part of apache#12857
@weizhouapache weizhouapache requested a review from Copilot March 19, 2026 20:00
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bhouse-nexthop
Copy link
Author

copilot issues, eh?

@codecov
Copy link

codecov bot commented Mar 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.70%. Comparing base (e93ae1a) to head (556ae05).

❗ There is a different number of reports uploaded between BASE (e93ae1a) and HEAD (556ae05). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (e93ae1a) HEAD (556ae05)
unittests 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               4.22   #12859       +/-   ##
=============================================
- Coverage     17.61%    3.70%   -13.91%     
=============================================
  Files          5917      448     -5469     
  Lines        531430    38028   -493402     
  Branches      64973     7036    -57937     
=============================================
- Hits          93586     1409    -92177     
+ Misses       427288    36432   -390856     
+ Partials      10556      187    -10369     
Flag Coverage Δ
uitests 3.70% <ø> (ø)
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Private Gateway static route with nexthop is not modifying PBR routing table

3 participants