3.1. The Poisson Equation in One Dimension
Considering only one space variable,
x, Equation (
1) can be written as:
where
Ω is the solution domain defining the possible
x values. A set of
N discrete
x values
𝑥1,𝑥2,⋯,𝑥𝑁 of
x is selected where the equation is to be solved. Then, the domain is
Ω=[𝑥1,𝑥𝑁]. The boundary of the domain is assumed to be the outer edges:
∂Ω={𝑥1,𝑥𝑁}. Some texts use the notation
𝑥0 and
𝑥𝑁+1 to indicate the boundary points and exclude them from the definition of the solution domain. However, in this paper, the edge points of the solution domain are taken as boundary points unless otherwise stated.
The domain points are referred to as 𝑥𝑖, 𝑖=1,2,⋯,𝑁. For simplicity, it is assumed that these points are separated by a constant number Δ𝑥 (i.e., 𝑥𝑖+1−𝑥𝑖=Δ𝑥,∀𝑖). The corresponding values of f and u are denoted 𝑓𝑖 and 𝑢𝑖, respectively (i.e., 𝑓(𝑥𝑖)=𝑓𝑖, 𝑢(𝑥𝑖)=𝑢𝑖). It is convenient to express these sets of discrete points as vectors 𝐱=[𝑥1𝑥2⋯𝑥𝑁]𝖳, 𝐟=[𝑓1𝑓2⋯𝑓𝑁]𝖳, and 𝐮=[𝑢1𝑢2⋯𝑢𝑁]𝖳. Note that transpose notations were used to write the column vectors in a compact manner. With vector notation, the differential equation can be imagined as a linear algebra problem where the vector 𝐮 is to be solved for a given 𝐟. The next step is to convert the second-derivative operator into a matrix operator so that a complete linear algebraic system can be formed.
To convert a continuous derivative operator to a discrete matrix operator, the starting point is the finite difference formulas for calculating the derivatives. The vectors
𝐮′=[𝑢′1𝑢′2⋯𝑢′𝑁]𝖳 and
𝐮″=[𝑢″1𝑢″2⋯𝑢″𝑁]𝖳 are defined to denote the first and second derivatives, respectively, corresponding to the points
𝐱 (i.e.,
𝑢𝑖′ =
𝑢′(𝑥𝑖), and
𝑢𝑖″ =
𝑢″(𝑥𝑖)). Using the second-order center-difference formulas, the first and second derivatives can be expressed as [
16]:
Note that there are other higher-order finite difference formulas for the above derivatives. The widely used second-order center-difference formulas are selected for the current analysis. Now, from Equations (
5) and (
6), focusing only on three consecutive quantities
𝑢𝑖−1,
𝑢𝑖,
𝑢𝑖+1, the following matrix equation can be written:
Equations (
7) and (
8) work fine for
𝑖=2 to
𝑁−1. However, for the boundary points
𝑖=1 and
𝑖=𝑁, the subscripts overflow the bounds, requiring values of
𝑢0 and
𝑢𝑁+1, respectively. Thus, the equation is not valid for those two cases (note that the outer boundary points are
𝑥1 and
𝑥𝑁). We resort to using forward (for
𝑖=1) and backward (for
𝑖=𝑁) finite difference formulas for these two cases [
16]:
Now, using Equations (
7) to (
12), the first and second derivatives at all points in the solution space can be calculated. These equations for all
i values can be compiled to form the following matrix equations:
It should be noted that for nonuniform grids, the term
Δ𝑥 would no longer be constant. As a result, in the place of the common
Δ𝑥 factor in the matrices, every matrix element would have a denominator corresponding to its own
Δ𝑥 value. Every other step discussed in this manuscript would remain unchanged. With the help of these matrices, the derivative operations can be expressed in linear algebra form as:
Here
𝐷𝑥 and
𝐷2𝑥 are
𝑁×𝑁 matrices, as shown in Equations (
13) and (
14). These are referred to as
matrix operators for the first and second derivatives, respectively, as they operate on a vector to calculate its derivatives. Equation (
4) can be converted to a discrete linear algebraic system using these matrix operators:
Now the boundary conditions must be implemented in such a way that they can be easily integrated with Equation (
17).
Let us consider a general implementation of boundary conditions in matrix form:
Here
B is referred to as the
boundary operator matrix, and
𝐠 is the known/given boundary function vector. For a given problem, the boundary operator matrix is determined by the type of the boundary conditions. For Dirichlet boundary conditions, the boundary function values are directly assigned to the boundary points. In such a case,
B would be an identity matrix, and
𝐠 would list the values of
𝐮 at the boundary. For Neumann boundary conditions, the value of the first derivative is assigned at the boundary points. For this case,
B would be the first-derivative matrix operator
𝐷𝑥, and
𝐠 would list the values of
𝐮′ at the boundary. Thus,
Here
𝐼𝑁 is the
𝑁×𝑁 identity matrix, and
𝐷𝑥 is expressed in Equation (
13). Note that although Equation (
19) relates the quantities in such a way that
B operates on the entire
𝐮, the equation is valid only at the boundary
∂Ω. In fact,
𝐠 elements at non-boundary points are unknown. In other words, the
locations of the boundaries are not defined explicitly within
B. However, to understand which entries of
B come from
𝐼𝑁 and which entries come from
𝐷𝑥, information about the geometry of the system is required. In the following paragraphs, we will discuss how to compile the linear algebraic system so that only the relevant rows of
B are used. This is where the remaining geometry information will be encoded. Before that, it is important to discuss how a multi-boundary system would be treated. As previously mentioned,
∂Ω can be subdivided into smaller regions
∂Ω1,∂Ω2⋯∂Ω𝑀 each with its own boundary conditions. Each
∂Ω𝑖 corresponds to a set of specific rows in the boundary operator matrix. For such cases,
𝐠 would be defined appropriately in a piecewise manner. For mixed boundary conditions, the rows of
B would be a compilation of rows taken from either
𝐼𝑁 or
𝐷𝑥 depending on which boundary regime the corresponding equation falls under. It should also be noted that for Neumann boundary conditions, an additional constraint such as Equation (
3) would be needed to obtain a unique solution.
After defining the matrix forms for the PDE and the boundary conditions, the process of integrating those into a single linear algebraic equation can be started. It is possible to simply solve Equation (
17) under the constraint defined in Equation (
19). However, an alternative and somewhat more elegant approach would be to form a single system matrix that incorporates both the PDE and the boundary conditions in a compact form. By observing the full form of the matrix and vectors in Equation (
17), it can be noted that each row of the matrix defines an equation containing a few elements of
𝐮. Since some of the elements of
𝐮 are on the boundary and follow Equation (
19), replacing the corresponding rows of
𝐷2𝑥 with those of
B can lead to the formation of a system matrix,
ℒ. This can be mathematically described as:
Of course, similar changes to the right-hand vector would need to be made as well. The
system right-hand vector,
𝐛, can be defined as:
According to this definition, if the
k-th element of
𝐮,
𝑢𝑘, corresponds to a Dirichlet boundary point, then the
k-th row of
ℒ would be the
k-th row of
𝐼𝑁. Further, the
k-th element of
𝐛 would be the
k-th row of
𝐠. All rows corresponding to non-boundary points will be identical to the rows of
𝐷2𝑥. Now, the system equation can finally be written as:
Dimension-wise,
ℒ is an
𝑁×𝑁 matrix, and
𝐛 is an
𝑁×1 vector. This is a fully defined linear system from which
𝐮 can be easily solved. Note that the derivative operator, the boundary conditions, and the problem geometry are all encoded within
ℒ and
b. Let us consider the 1D case with
𝑥1 and
𝑥𝑁 being the boundary points. For Dirichlet boundary conditions, this correspond to
[𝑢1𝑢𝑁]𝖳=[𝑔1𝑔𝑁]𝖳, where
𝑔1 and
𝑔𝑁 are given. Using Equations (
20) and (
21), the matrix system is constructed and solved. Note that
𝑔𝑖,∀𝑖≠1,𝑁, are not defined and are not needed to form
𝐛.
One notable feature of the proposed system matrix construction procedure is that part of it is problem independent. Note that only the B matrix and the 𝐠 vector depend on the boundary condition types at different boundaries. The 𝐷2𝑥 matrix is the same for all problems. The construction of ℒ involves swapping out certain rows of 𝐷2𝑥 with B. This part depends on the problem geometry (as points on ∂Ω need to be identified). Thus, modifying the solver for different geometries and boundary conditions involves selecting the appropriate form of B and modifying the row-swapping operation of ℒ only (the same operation must be done for the right-hand vector, also). This is a major benefit over conventional procedures for which the system matrix is constructed from scratch for every geometry/boundary condition.
It is obvious that the size of the matrix operators, having size 𝑁×𝑁, can be very large for moderately large values of N. As will be discussed later, the sizes of the matrices are even larger for cases with higher dimensions. Working with such large matrices in full form can be computationally demanding. Fortunately, these matrices are sparse in nature (i.e., the number of non-zero elements is small compared to the full size of the matrix), as can be seen from the forms of 𝐷𝑥 and 𝐷2𝑥. As such, when solving numerically, they are always defined as sparse matrices, which significantly reduces memory requirements and speeds up the operation. Almost all programming languages commonly used for numerical analysis have sparse-matrix libraries that can be used for this (e.g., Python has scipy.sparse package).
3.2. The Poisson Equation in Two Dimensions
The 2D Poisson equation in a Cartesian
𝑥𝑦 plane is given by:
Here the solution
𝑢=𝑢(𝑥,𝑦) spans a 2D space. The matrix analysis for the 1D Poisson equation discussed in
Section 3.1 can be extended for this 2D case. The matrix operators developed for the 1D case can be used as the building blocks. The key idea is to convert the 2D problem into an equivalent 1D problem. This is done by mapping the 2D solution grid into a 1D vector, solving the 1D vector, and then remapping it to the original 2D space. The main tasks are developing the mapping algorithm, figuring out how the boundary conditions are mapped from 2D to 1D space, and assembling the corresponding matrix operators.
The 2D solution space in the 𝑥𝑦-Cartesian plane is defined as Ω=[𝑥1,𝑥𝑁𝑥]×[𝑦1,𝑦𝑁𝑦]. This denotes a continuous rectangular region with x values spanning from 𝑥1 to 𝑥𝑁𝑥 and y values spanning from 𝑦1 to 𝑦𝑁𝑦. Note that the analysis would hold for any shape of solution space. In discrete space, a grid of equally spaced 𝑁𝑦×𝑁𝑥 points (𝑦𝑘,𝑥𝑗) are considered, where 𝑗=1,2,⋯,𝑁𝑥 and 𝑘=1,2,⋯,𝑁𝑦. The x and y spacing are denoted Δ𝑥 and Δ𝑦, respectively. These points can be imagined as elements of a matrix with 𝑁𝑦 rows and 𝑁𝑥 columns, as shown in . Any point can be addressed by the matrix coordinate (𝑘,𝑗), where k refers to the row number and j refers to the column number. The solution u at point (𝑦𝑘,𝑥𝑗) is denoted as 𝑢(𝑘,𝑗)=𝑢𝑘𝑗. The order of the indices j and k is selected in this way to be consistent with syntax of common numerical coding languages (e.g., Python and MATLAB).
Figure 1.
Process of mapping a 2D grid to a 1D vector and vice-versa.
Now,
𝑢𝑘𝑗, being 2D, is not an ideal input for a matrix operator. The elements of the 2D grid are rearranged by stacking the columns on top of each other to form one large
𝑁𝑥𝑁𝑦×1 vector, referred to as
𝐮1𝐷. This is shown in [
17]. Each element of this
𝐮1𝐷 corresponds to a specific grid point
(𝑦𝑘,𝑥𝑗) and the corresponding solution
𝑢𝑘𝑗. The elements of
𝐮1𝐷 can be addressed by a single index
𝑖=1,2,⋯,𝑁𝑥𝑁𝑦. Note that there are many other valid approaches to mapping a 2D matrix into an 1D vector. This discussion is limited to the column-stacking method, which is relatively simple and intuitive. For this mapping, the 2D and 1D indices are related as:
The opposite transformation is:
Here mod refers to the modulo operation. Using these relationships, it is possible to transfer back and forth between the 2D representation and the equivalent 1D representation. This process is referred to as
mapping. The mapping process is shown in .
After obtaining
𝐮1𝐷 from the mapping process, the corresponding derivative matrix operators need to formulated. The matrix operators for the first and second partial derivatives with respect to
x are expressed as
𝐷2𝐷𝑥 and
𝐷2𝐷2𝑥, respectively. Similar quantities with respect to
y are expressed as
𝐷2𝐷𝑦 and
𝐷2𝐷2𝑦, respectively. The superscripts denote that these are 2D matrix operators. For the 1D case discussed in
Section 3.1, the consecutive elements of the vector
𝐮 corresponded to consecutive points along a 1D spatial axis. For the 2D case, however, the relation between the elements of
𝐮1𝐷 and the corresponding spatial points is more complex. Every
𝑁𝑦 block of elements in
𝐮1𝐷 (shown in different colors in ) represents a column in the 2D representation. They have consecutive
y values for a given
x value. Within this block, matrix operators of similar form as Equation (
13) and (
14) would work for the
y derivative. As one moves from one of these blocks to the next (moving from one color to another in ), the
y values restart from the beginning and the
x values increment by one step. Our
x derivative operator must operate on this jumbled space and accurately pick out the correct elements for a differentiation operation.
First, the
y derivatives are considered, as they are simpler to understand than the
x derivatives for this mapping process. The
y partial derivatives in vector form are denoted
𝐮𝑦,1𝐷 and
𝐮2𝑦,1𝐷. The 2D matrix operator for the
y partial derivative must be such that it sifts each
𝑁𝑦 block of elements and multiplies it by the 1D-derivative matrix operator. As it goes through each of the columns, the results should be stacked on a column to maintain the same spatial mapping as
𝐮1𝐷. This can be done by forming a
block diagonal matrix whose diagonal blocks are copies of the 1D
𝐷𝑦 matrix operator [
8,
17] (
𝐷𝑦 has the same form as
𝐷𝑥 from Equation (
13), with
Δ𝑥 replaced by
Δ𝑦). The process is shown graphically in . Each rectangular block in
𝐷2𝐷𝑦 is an
𝑁𝑦×𝑁𝑦 matrix. The zero blocks are zero matrices of the same size. Each block operates on an
𝑁𝑦×1 sized block of
𝐮1𝐷 (represented as different-colored columns and labeled
𝐶𝑜𝑙1,𝐶𝑜𝑙2,⋯,𝐶𝑜𝑙𝑁𝑥 ). This is shown in the figure by connecting a
𝐷2𝐷𝑦 matrix block with the corresponding
𝐮1𝐷 column block by curved lines. For a given block-row (the rows drawn in the figure with black lines) of the matrix, all elements of
𝐮1𝐷 beside a specific block of
𝑁𝑦 consecutive elements are ignored (i.e., multiplied by zeros). Thus, the derivative operation is performed for each of these column blocks as one moves from one block-row of the matrix to another. This matrix multiplication produces a column vector representing the
y derivatives mapped in the same format as
𝐮1𝐷. The second-derivative matrix,
𝐷2𝐷2𝑦, can be similarly constructed by replacing
𝐷𝑦 with
𝐷2𝑦 in each block.
Figure 2.
Matrix first-derivative operator (with respect to
y) for the two-dimensional case.
𝐷𝑦 has the same form as
𝐷𝑥 which is defined in Equation (
13).
Let us now consider the matrix operators for the
x partial derivatives,
𝐷2𝐷𝑥 and
𝐷2𝐷2𝑥. When multiplied with
𝐮1𝐷, these matrices produce the partial derivatives in vector form denoted
𝐮𝑥,1𝐷 and
𝐮2𝑥,1𝐷, respectively. First, the locations of the elements representing consecutive
x values are identified and listed in the 1D-mapped vectors. The
x values increase by one when moving from left to right in a given row (move by one column) in the 2D representation (). In the 1D map, movement of one space along a row translates into movement by
𝑁𝑦 elements in the corresponding 1D-mapped vector. A block matrix construction is again used to form
𝐷2𝐷𝑥 from
𝐷𝑥. This time, however, each row of
𝐷𝑥 must operate on elements of
𝐮1𝐷 that are
𝑁𝑦 spaces apart. This can be done by adding
𝑁𝑦 zeros after each element in a row of
𝐷𝑥 and then copying and arranging the rows appropriately [
17]. The required configuration is shown in . The elements of
𝐷𝑥 are repeated in the diagonal of each block in
𝐷2𝐷𝑥. Thus, each block is a diagonal matrix. Each block sifts out certain elements of
𝐮1𝐷, as shown by the curved lines in the figure. Careful observation shows that these elements are indeed the ones required to calculate the derivatives at that point. A similar block matrix can be constructed for
𝐷2𝐷2𝑥 by replacing
𝐷𝑥 elements with
𝐷2𝑥 elements.
Figure 3.
Matrix first-derivative operator (with respect to
x) for the two-dimensional case.
𝐷𝑥(𝑗,𝑘) refers to the element from row
j and column
k of the
𝐷𝑥 matrix defined in Equation (
13).
Now that the matrix operators have been defined, their construction using simple mathematical operations is discussed. A powerful tool in constructing block matrices is the Kronecker product [
4,
18]. The Kronecker product between two matrices
A (size
𝑝×𝑞) and
B (size
𝑚×𝑛) is defined as:
where
𝐴𝑖𝑗 represents the
(𝑖,𝑗) element of matrix
A, and ⊗ represents Kronecker product operation. Note that each term in Equation (
27) is a matrix itself. The
𝐴⊗𝐵 block matrix is formed by tiling
B matrices after multiplying them with an element of the
A matrix. It is not difficult to see that
𝐷2𝐷𝑦 can be constructed from the Kronecker product of
𝐷𝑦 and an identity matrix of size
𝑁𝑥 (denoted as
𝐼𝑁𝑥):
Careful observation of and Equation (
27) suggests that
𝐷2𝐷𝑥 can also be computed using the Kronecker product of
𝐷𝑥 and an identity matrix of size
𝑁𝑦×𝑁𝑦 (denoted
𝐼𝑁𝑦):
The second-derivative matrices are calculated identically:
The overall sizes of these matrices are
𝑁𝑥𝑁𝑦×𝑁𝑥𝑁𝑦. This can be very large even for moderate values of
𝑁𝑥 and
𝑁𝑦. However, it is noted that the sparsity of the matrices from the 1D case is maintained in the 2D case as well. Even more so than with the 1D case, any numerical implementation of the 2D Poisson equation should employ sparse-matrix algorithms.
The step after constructing the differentiation operators is formulation of the boundary operator matrix. The 2D boundary operator matrix,
𝐵2𝐷, would be identical to the one discussed in the previous section for the one-dimensional case. Equation (
19) would be slightly modified to accommodate for the larger two-dimensional matrices:
where
𝐼𝑁𝑥𝑁𝑦 is an identity matrix of size
𝑁𝑥𝑁𝑦×𝑁𝑥𝑁𝑦. As there are two independent variables, there are two possible Neumann boundary conditions that can be applied. Which derivative to use for the Neumann boundary is determined by the problem statement. Having defined the boundary operator, construction of the system matrix operator is straightforward:
The right-hand vector also follows a similar definition as the 1D case:
where
𝐟1𝐷 and
𝐠1𝐷 are 1D-mapped versions of the corresponding two-dimensional quantities,
𝐟(𝑦𝑘,𝑥𝑗) and
𝐠(𝑦𝑘,𝑥𝑗), respectively. Obviously, the algorithm used for the 1D mapping here should be identical to the one used for calculating
𝐮1𝐷. Finally, the system equation is constructed as:
Just like the one-dimensional case, this is a simple linear algebraic equation that can be solved easily. It should be noted that the solution is in a 1D-mapped vector form. Thus, an additional step of converting it back to the 2D form is necessary. This can be done using Equations (
25) and (
26).
It should be noted that only the 𝐵2𝐷 matrix, the 𝐠1𝐷 vector, and the construction of ℒ and 𝐛 are geometry and boundary condition dependent. The rest of the matrices and vectors are problem independent. Thus, configuring the solver for different geometries and boundary conditions is straightforward.