Merge pull request #3193 from verilog-to-routing/reconstruction_grids_with_LegalizationClusterId

FlatRecon: Flat Placement Reconstruction Full Legalizer

Author: vaughnbetz

.github/workflows/nightly_test_manual.yml

@@ -51,6 +51,7 @@ jobs:
         ./dev/upgrade_vtr_archs.sh
         make get_symbiflow_benchmarks
         make get_zeroasic_rr_graphs
+        make get_flat_placements
 
     # Build VTR using the default build options.
     - name: 'Build VTR'

CMakeLists.txt

@@ -368,6 +368,14 @@ add_custom_target(get_zeroasic_rr_graphs
     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
     COMMENT "Downloading (~2MB) and extracting Zero ASIC RR graphs (~0.1GB) into VTR source tree.")
 
+#
+# Flat Placement Files
+#
+add_custom_target(get_flat_placements
+    COMMAND ./vtr_flow/scripts/get_flat_placement_files.py --vtr_flow_dir ./vtr_flow
+    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+    COMMENT "Downloading (~5MB) and extracting Flat Placement Files (~0.1GB) into VTR source tree.")
+
 #
 # Unit Testing
 #

doc/src/vpr/command_line_usage.rst

@@ -444,22 +444,22 @@ Use the options below to override this default naming behaviour.
     Reads a file containing the locations of each atom on the FPGA.
     This is used by the packer to better cluster atoms together.
 
-    The flat placement file (which often ends in ``.fplace``) is a text file
+    The flat placement file (which often ends in :ref:`.fplace <vpr_flat_place_file>`) is a text file
     where each line describes the location of an atom. Each line in the flat
     placement file should have the following syntax:
 
     .. code-block:: none
 
-        <atom_name : str> <x : float> <y : float> <layer : float> <atom_sub_tile : int> <atom_site_idx? : int>
+        <atom_name : str> <x : float> <y : float> <layer : float> <atom_sub_tile : int>
 
     For example:
 
     .. code-block:: none
 
-        n523  6 8 0 0 3
-        n522  6 8 0 0 5
-        n520  6 8 0 0 2
-        n518  6 8 0 0 16
+        n523  6 8 0 0
+        n522  6 8 0 0
+        n520  6 8 0 0
+        n518  6 8 0 0
 
     The position of the atom on the FPGA is given by 3 floating point values
     (``x``, ``y``, ``layer``). We allow for the positions of atom to be not
@@ -475,25 +475,29 @@ Use the options below to override this default naming behaviour.
     the sub-tile of an atom is unkown (allowing the packing algorithm to choose
     any sub-tile at the given (x, y, layer) location).
 
-    The ``site_idx`` is an optional index into a linearized list of primitive
-    locations within a cluster-level block which may be used as a hint to
-    reconstruct clusters.
-
     .. warning::
 
         This interface is currently experimental and under active development.
 
 .. option:: --write_flat_place <file>
 
-    Writes the post-placement locations of each atom into a flat placement file.
+    Writes the post-placement locations of each atom into a flat placement file
+    (see :ref:`flat placement file format <vpr_flat_place_file>`).
+
+    For each atom in the netlist, the following information is stored into the
+    flat placement file:
+
+    * The x, y, and sub_tile location of the cluster that contains this atom.
+
+.. option:: --write_legalized_flat_place <file>
+
+    Writes the post-legalization locations of each atom into a flat placement file
+    (see :ref:`flat placement file format <vpr_flat_place_file>`).
 
     For each atom in the netlist, the following information is stored into the
     flat placement file:
 
     * The x, y, and sub_tile location of the cluster that contains this atom.
-    * The flat site index of this atom in its cluster. The flat site index is a
-      linearized ID of primitive locations in a cluster. This may be used as a
-      hint to reconstruct clusters.
 
 .. _netlist_options:
 
@@ -1287,14 +1291,20 @@ Analytical Placement is generally split into three stages:
 
     **Default:** ``bipartitioning``
 
-.. option:: --ap_full_legalizer {naive | appack}
+.. option:: --ap_full_legalizer {naive | appack | flat-recon}
 
     Controls which Full Legalizer to use in the AP Flow.
 
     * ``naive`` Use a Naive Full Legalizer which will try to create clusters exactly where their atoms are placed.
 
     * ``appack`` Use APPack, which takes the Packer in VPR and uses the flat atom placement to create better clusters.
 
+    * ``flat-recon`` Use the Flat Placement Reconstruction Full Legalizer which tries to reconstruct a clustered placement that is
+      as close to the incoming flat placement as possible. It can be used to read a flat placement from a :ref:`.fplace <vpr_flat_place_file>` file
+      or on the (in memory) output of VTR's integrated Global Placement algorithm. In both cases, it expects the given solution to be close to legal.
+      If used with a :ref:`.fplace <vpr_flat_place_file>` file, each atom in a molecule should have compatible location information. It is legal to
+      leave some molecules unconstrained; the reconstruction phase will choose where to place them but does not attempt to optimize these locations.
+
     **Default:** ``appack``
 
 .. option:: --ap_detailed_placer {none | annealer}

doc/src/vpr/file_formats.rst

@@ -696,6 +696,42 @@ Placement File Format Example
     xor5        1       2       0       0      #6
     [1]         1       1       0       1      #7
 
+.. _vpr_flat_place_file:
+
+Flat Placement File Format (.fplace)
+------------------------------------
+The flat placement file is a text file where each line describes the location of
+an atom. Each line in the flat placement file should have the following syntax:
+
+.. code-block:: none
+
+        <atom_name : str> <x : float> <y : float> <layer : float> <atom_sub_tile : int>
+
+For example:
+
+    .. code-block:: none
+
+        n523  6 8 0 0
+        n522  6 8 0 0
+        n520  6 9 0 0
+        n518  6 9 0 0
+
+The position of the atom on the FPGA is given by 3 floating point values
+(``x``, ``y``, ``layer``). We allow x and y locations to be off-grid since
+this flat placement will be fed into the packer and placer, which will snap
+the positions to grid locations. For 2D FPGA architectures, the ``layer`` should be 0.
+
+The ``sub_tile`` is a clustered placement construct: which cluster-level
+location at a given (x, y, layer) should these atoms go at (relevant when
+multiple clusters can be stacked there). A sub-tile of -1 may be used when
+the sub-tile of an atom is unkown (allowing the packing algorithm to choose
+any sub-tile at the given (x, y, layer) location).
+
+When used with ``flat-recon`` full legalizer (see :option:`vpr --ap_full_legalizer`),
+each atom in a molecule should have compatible location information. It is legal to
+leave some molecules unconstrained; the reconstruction phase will choose where
+to place them but does not attempt to optimize these locations.
+
 .. _vpr_route_file:
 
 Routing File Format (.route)

libs/libarchfpga/src/physical_types.h

@@ -890,7 +890,40 @@ struct t_physical_tile_loc {
     operator bool() const {
         return !(x == ARCH_FPGA_UNDEFINED_VAL || y == ARCH_FPGA_UNDEFINED_VAL || layer_num == ARCH_FPGA_UNDEFINED_VAL);
     }
+
+    /**
+     * @brief Comparison operator for t_physical_tile_loc
+     *
+     * Tiles are ordered first by layer number, then by x, and finally by y.
+     */
+    friend bool operator<(const t_physical_tile_loc& lhs, const t_physical_tile_loc& rhs) {
+        if (lhs.layer_num != rhs.layer_num)
+            return lhs.layer_num < rhs.layer_num;
+        if (lhs.x != rhs.x)
+            return lhs.x < rhs.x;
+        return lhs.y < rhs.y;
+    }
+
+    friend bool operator==(const t_physical_tile_loc& a, const t_physical_tile_loc& b) {
+        return a.x == b.x && a.y == b.y && a.layer_num == b.layer_num;
+    }
+
+    friend bool operator!=(const t_physical_tile_loc& a, const t_physical_tile_loc& b) {
+        return !(a == b);
+    }
+};
+
+namespace std {
+template<>
+struct hash<t_physical_tile_loc> {
+    std::size_t operator()(const t_physical_tile_loc& v) const noexcept {
+        std::size_t seed = std::hash<int>{}(v.x);
+        vtr::hash_combine(seed, v.y);
+        vtr::hash_combine(seed, v.layer_num);
+        return seed;
+    }
 };
+} // namespace std
 
 /** Describes I/O and clock ports of a physical tile type
  *

vpr/src/analytical_place/analytical_placement_flow.cpp

@@ -18,6 +18,7 @@
 #include "gen_ap_netlist_from_atoms.h"
 #include "global_placer.h"
 #include "globals.h"
+#include "load_flat_place.h"
 #include "netlist_fwd.h"
 #include "partial_legalizer.h"
 #include "partial_placement.h"
@@ -80,6 +81,7 @@ static void print_ap_netlist_stats(const APNetlist& netlist) {
  * to be generated on ap_netlist or have the same blocks.
  */
 static void convert_flat_to_partial_placement(const FlatPlacementInfo& flat_placement_info, const APNetlist& ap_netlist, const Prepacker& prepacker, PartialPlacement& p_placement) {
+    size_t num_mols_assigned_to_center = 0;
     for (APBlockId ap_blk_id : ap_netlist.blocks()) {
         // Get the molecule that AP block represents
         PackMoleculeId mol_id = ap_netlist.block_molecule(ap_blk_id);
@@ -97,6 +99,8 @@ static void convert_flat_to_partial_placement(const FlatPlacementInfo& flat_plac
             float current_loc_layer = flat_placement_info.blk_layer[atom_blk_id];
             int current_loc_sub_tile = flat_placement_info.blk_sub_tile[atom_blk_id];
             if (found_valid_atom) {
+                if (current_loc_x == -1 || current_loc_y == -1)
+                    continue;
                 if (current_loc_x != atom_loc_x || current_loc_y != atom_loc_y || current_loc_layer != atom_loc_layer || current_loc_sub_tile != atom_loc_sub_tile)
                     VPR_FATAL_ERROR(VPR_ERROR_AP,
                                     "Molecule of ID %zu contains atom %s (ID: %zu) with a location (%g, %g, layer: %g, subtile: %d) "
@@ -105,21 +109,53 @@ static void convert_flat_to_partial_placement(const FlatPlacementInfo& flat_plac
                                     current_loc_x, current_loc_y, current_loc_layer, current_loc_sub_tile,
                                     atom_loc_x, atom_loc_y, atom_loc_layer, atom_loc_sub_tile);
             } else {
-                atom_loc_x = current_loc_x;
-                atom_loc_y = current_loc_y;
-                atom_loc_layer = current_loc_layer;
-                atom_loc_sub_tile = current_loc_sub_tile;
-                found_valid_atom = true;
+                if (current_loc_x != -1 && current_loc_y != -1) {
+                    atom_loc_x = std::clamp(current_loc_x, 0.0f,
+                                            static_cast<float>(g_vpr_ctx.device().grid.width() -1));
+                    atom_loc_y = std::clamp(current_loc_y, 0.0f,
+                                            static_cast<float>(g_vpr_ctx.device().grid.height() -1));
+                    // If current_loc_layer or current_loc_sub_tile are unset (-1), default to layer 0 and sub_tile 0.
+                    if (current_loc_layer == -1)
+                        current_loc_layer = 0;
+                    if (current_loc_sub_tile == -1)
+                        current_loc_sub_tile = 0;
+                    atom_loc_layer = current_loc_layer;
+                    atom_loc_sub_tile = current_loc_sub_tile;
+                    found_valid_atom = true;
+                }
             }
         }
-        // Ensure that there is a valid atom in the molecule to pass its location.
-        VTR_ASSERT_MSG(found_valid_atom, "Each molecule must contain at least one valid atom");
-        // Pass the placement information
-        p_placement.block_x_locs[ap_blk_id] = atom_loc_x;
-        p_placement.block_y_locs[ap_blk_id] = atom_loc_y;
-        p_placement.block_layer_nums[ap_blk_id] = atom_loc_layer;
-        p_placement.block_sub_tiles[ap_blk_id] = atom_loc_sub_tile;
+        // If any atom in the molecule has a location assigned, use that location
+        // for the entire AP block. Otherwise, assign the AP block to the center
+        // of the device grid and update the flat placement info for all its atoms accordingly.
+        if (!found_valid_atom) {
+            num_mols_assigned_to_center++;
+            VTR_LOG_WARN("No atoms of molecule ID %zu provided in the flat placement. Assigning it to the device center.\n", mol_id);
+            p_placement.block_x_locs[ap_blk_id] = g_vpr_ctx.device().grid.width() / 2.0f;
+            p_placement.block_y_locs[ap_blk_id] = g_vpr_ctx.device().grid.height() / 2.0f;
+            p_placement.block_layer_nums[ap_blk_id] = 0;
+            p_placement.block_sub_tiles[ap_blk_id] = 0;
+            // Update flat placement for atoms of that molecule accordingly.
+            // Needed for flat placement reconstruction statistics reporting.
+            for (AtomBlockId atom_blk_id : mol.atom_block_ids) {
+                g_vpr_ctx.mutable_atom().mutable_flat_placement_info().blk_x_pos[atom_blk_id] = g_vpr_ctx.device().grid.width() / 2.0f;
+                g_vpr_ctx.mutable_atom().mutable_flat_placement_info().blk_y_pos[atom_blk_id] = g_vpr_ctx.device().grid.height() / 2.0f;
+                g_vpr_ctx.mutable_atom().mutable_flat_placement_info().blk_layer[atom_blk_id] = 0;
+                g_vpr_ctx.mutable_atom().mutable_flat_placement_info().blk_sub_tile[atom_blk_id] = 0;
+            }
+            // TODO: If an atom's location is specified in the placement constraints,
+            //       verify it matches the assigned flat placement. If not, override the
+            //       flat placement with the constraint location and warn the user.
+        } else {
+            // Pass the placement information
+            p_placement.block_x_locs[ap_blk_id] = atom_loc_x;
+            p_placement.block_y_locs[ap_blk_id] = atom_loc_y;
+            p_placement.block_layer_nums[ap_blk_id] = atom_loc_layer;
+            p_placement.block_sub_tiles[ap_blk_id] = atom_loc_sub_tile;
+        }
     }
+    VTR_LOG("%zu of %zu molecules placed at device center (no atoms of these molecules found in flat placement).\n",
+        num_mols_assigned_to_center, ap_netlist.blocks().size());
 }
 
 /**
@@ -245,6 +281,15 @@ void run_analytical_placement_flow(t_vpr_setup& vpr_setup) {
     // Print the device utilization
     print_device_utilization(target_device_utilization);
 
+    // Write out a flat placement file at the end of Full Legalization if the
+    // option is specified.
+    if (!vpr_setup.FileNameOpts.write_legalized_flat_place_file.empty()) {
+        write_flat_placement(vpr_setup.FileNameOpts.write_legalized_flat_place_file.c_str(),
+                             g_vpr_ctx.clustering().clb_nlist,
+                             g_vpr_ctx.placement().block_locs(),
+                             g_vpr_ctx.clustering().atoms_lookup);
+    }
+
     // Run the Detailed Placer.
     std::unique_ptr<DetailedPlacer> detailed_placer = make_detailed_placer(ap_opts.detailed_placer_type,
                                                                            g_vpr_ctx.placement().blk_loc_registry(),

vpr/src/analytical_place/ap_flow_enums.h

@@ -39,9 +39,9 @@ enum class e_ap_partial_legalizer {
  * enum can select between these different Full Legalizers.
  */
 enum class e_ap_full_legalizer {
-    Naive,                ///< The Naive Full Legalizer, which clusters atoms placed in the same tile and tries to place them in that tile according to the flat placement.
-    APPack,               ///< The APPack Full Legalizer, which uses the flat placement to improve the Packer and Placer.
-    Basic_Min_Disturbance ///< The Basic Min. Disturbance Full Legalizer, which tries to reconstruct a clustered placement that is as close to the incoming flat placement as it can.
+    Naive,    ///< The Naive Full Legalizer, which clusters atoms placed in the same tile and tries to place them in that tile according to the flat placement.
+    APPack,   ///< The APPack Full Legalizer, which uses the flat placement to improve the Packer and Placer.
+    FlatRecon ///< The Flat Placement Reconstruction Full Legalizer, which tries to reconstruct a clustered placement (or the in-memory global placement) that is as close to the incoming flat placement as it can.
 };
 
 /**