Skip to main content

Database Integration with MongoDB

In this tutorial, we'll learn how to implement a CRUD API with MongoDB database integration using the Gonyx framework's contrib/mongokit package which wraps around the MongoDB Go Driver.

What You'll Learn​

  • How to set up database models with MongoDB in a Gonyx application
  • How to create database operations for CRUD functionality
  • How to implement controllers that interact with MongoDB models
  • How to initialize MongoDB connections in your application

Prerequisites​

Before starting this tutorial, make sure you've:

  • Completed the Quick Start tutorial
  • Basic understanding of Go programming
  • Familiarity with MongoDB concepts
  • Go 1.23.0 or later installed
  • MongoDB instance running (locally or in the cloud)

Project Setup​

We'll build on the project we created in the Quick Start tutorial. If you haven't completed that tutorial, please do so first.

tip

First, create a new Gonyx project using the CLI:

gonyx init gonyx-mongo-db --path .
cd gonyx-mongo-db

Step 1: Configure MongoDB Connection​

First, we need to configure the MongoDB connection in our application. We'll need to update two configuration files:

  1. First, add the MongoDB module to your base configuration:
configs/dev/base.json
{
  "name": "gonyx-mongo-db",
  "config_must_watched": true,
  "config_remote_addr": "0.0.0.0:7777",
  "config_remote_infra": "grpc",
  "config_remote_duration": 300,
  "modules": [
    {"name":"logger", "type": "local"},
    {"name":"http", "type": "local"},
    {"name":"protobuf", "type": "local"},
    {"name":"mongodb", "type": "local"}
  ]
}
  1. Then create a MongoDB configuration file:
configs/dev/mongodb.json
{
  "connections": ["server1"],
  "server1": {
    "uri": "mongodb://localhost:27017",
    "auth": {
      "mechanism": "SCRAM-SHA-256",
      "username": "",
      "password": "",
      "source": "admin"
    },
    "options": {
      "max_pool_size": 100,
      "min_pool_size": 0,
      "max_conn_idle_time": 0,
      "connect_timeout": 30000
    }
  }
}

Understanding the MongoDB Configuration​

The Gonyx MongoDB configuration has several important sections:

  1. connections: An array listing which server configurations should be enabled. In our example, we're only using server1.

  2. MongoDB Server Configuration:

    • Each server (like server1) has its own configuration block
    • uri: The MongoDB connection URI
    • auth: Authentication options (mechanism, username, password, source)
    • options: Connection pool options

For a production environment, you'd want to use secure credentials and potentially a connection string to a MongoDB Atlas cluster or other MongoDB deployment.

Step 2: Create the Database Model​

Next, let's create a new file called model.go in the app directory to define our MongoDB data model. For MongoDB, we'll need to define our document structure with appropriate BSON tags for MongoDB serialization:

app/model.go
/*
Create By Gonyx Framework

Copyright © 2025
Project: gonyx-mongo-db
File: "app/model.go"
------------------------------
*/

package app

import (
	"time"

	"go.mongodb.org/mongo-driver/bson/primitive"
)

// MongoDB related constants
const (
	// MongoDB instance name defined in the config
	MongoDBInstance = "server1"
	DatabaseName    = "gonyx_demo"
	UsersCollection = "users"
)

// User represents a user document in MongoDB
type User struct {
	ID        primitive.ObjectID `bson:"_id,omitempty" json:"id"`
	Name      string             `bson:"name" json:"name"`
	Email     string             `bson:"email" json:"email"`
	Age       int                `bson:"age" json:"age"`
	CreatedAt time.Time          `bson:"created_at" json:"created_at"`
	UpdatedAt time.Time          `bson:"updated_at" json:"updated_at"`
}

Model Definition​

Let's break down the key components of our MongoDB model:

  1. Document Structure: We define a User struct with BSON and JSON tags for MongoDB serialization and API responses:

    • ID: The MongoDB ObjectID with _id as the BSON field name (MongoDB's native primary key)
    • Name, Email, Age: Basic user properties
    • CreatedAt, UpdatedAt: Timestamps for tracking document changes

  2. MongoDB Constants: We define constants for:

    • The MongoDB instance name from our configuration
    • The database name
    • The collection name where user documents will be stored

Unlike SQL databases, MongoDB doesn't need schema migration since it's schema-less by nature. However, we still define a structured Go type to work with our documents in a type-safe manner.

Step 3: Implementing CRUD Operations​

Now that we have our data model in place, let's implement each CRUD operation one by one. First, update your model.go file to add the necessary imports and implement the CREATE operation:

app/model.go (Add imports and CREATE operation)
import (
	"context"
	"errors"
	"fmt"
	"time"

	"github.com/Blocktunium/gonyx/contrib/mongokit"
	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/bson/primitive"
	"go.mongodb.org/mongo-driver/mongo"
	"go.mongodb.org/mongo-driver/mongo/options"
)

// CreateUser inserts a new user into the database
func CreateUser(user User) (*User, error) {
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	// Set creation and update times
	now := time.Now()
	user.CreatedAt = now
	user.UpdatedAt = now

	// Get MongoDB instance from mongokit manager
	db, err := mongokit.GetManager().GetMongoDb(MongoDBInstance)
	if err != nil {
		return nil, fmt.Errorf("failed to get MongoDB instance: %w", err)
	}

	// Insert the document
	result, err := db.Collection(UsersCollection).InsertOne(ctx, user)
	if err != nil {
		return nil, err
	}

	// Get the ID of the inserted document
	if oid, ok := result.InsertedID.(primitive.ObjectID); ok {
		user.ID = oid
	}

	return &user, nil
}

Understanding the CREATE Operation​

Let's examine what's happening in this function:

  1. We create a context with a timeout to ensure our operation doesn't hang indefinitely
  2. We set the creation and update timestamps for the new document
  3. We retrieve the MongoDB database instance from the mongokit manager
  4. We insert the document into the users collection
  5. We extract the generated ObjectID from the operation result and set it on our User struct
  6. We return the complete user with the generated ID

Implementing READ Operations​

Next, let's add the READ operations to retrieve users by ID and email:

app/model.go (Add READ operations)
// GetUserByID retrieves a user by ID
func GetUserByID(id string) (*User, error) {
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	// Convert string ID to ObjectID
	objectID, err := primitive.ObjectIDFromHex(id)
	if err != nil {
		return nil, fmt.Errorf("invalid ID format: %v", err)
	}

	// Get MongoDB instance from mongokit manager
	db, err := mongokit.GetManager().GetMongoDb(MongoDBInstance)
	if err != nil {
		return nil, fmt.Errorf("failed to get MongoDB instance: %w", err)
	}

	// Find the user
	filter := bson.M{"_id": objectID}
	var user User

	err = db.Collection(UsersCollection).FindOne(ctx, filter).Decode(&user)
	if err != nil {
		if errors.Is(err, mongo.ErrNoDocuments) {
			return nil, nil // No user found
		}
		return nil, err
	}

	return &user, nil
}

// GetUserByEmail retrieves a user by email
func GetUserByEmail(email string) (*User, error) {
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	// Get MongoDB instance from mongokit manager
	db, err := mongokit.GetManager().GetMongoDb(MongoDBInstance)
	if err != nil {
		return nil, fmt.Errorf("failed to get MongoDB instance: %w", err)
	}

	// Find the user by email
	filter := bson.M{"email": email}
	var user User

	err = db.Collection(UsersCollection).FindOne(ctx, filter).Decode(&user)
	if err != nil {
		if errors.Is(err, mongo.ErrNoDocuments) {
			return nil, nil // No user found
		}
		return nil, err
	}

	return &user, nil
}

Implementing LIST Operation​

Next, let's add the LIST operation to retrieve multiple users with pagination support:

app/model.go (Add LIST operation)
// ListUsers retrieves all users with optional pagination
func ListUsers(limit, skip int) ([]*User, error) {
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()

	// Get MongoDB instance from mongokit manager
	db, err := mongokit.GetManager().GetMongoDb(MongoDBInstance)
	if err != nil {
		return nil, fmt.Errorf("failed to get MongoDB instance: %w", err)
	}

	opts := options.Find()
	if limit > 0 {
		opts.SetLimit(int64(limit))
	}
	if skip > 0 {
		opts.SetSkip(int64(skip))
	}
	opts.SetSort(bson.M{"created_at": -1}) // Sort by creation time, newest first

	// Find all users with the given options
	cursor, err := db.Collection(UsersCollection).Find(ctx, bson.M{}, opts)
	if err != nil {
		return nil, err
	}
	defer cursor.Close(ctx)

	var users []*User
	if err = cursor.All(ctx, &users); err != nil {
		return nil, err
	}

	return users, nil
}

Understanding the LIST Operation​

The ListUsers function demonstrates several MongoDB cursor features:

  1. We use the Find() method to get multiple documents at once
  2. We set pagination options with SetLimit and SetSkip
  3. We sort the results by creation time in descending order
  4. We use cursor.All() to collect all matching documents into a slice
  5. We properly manage the cursor lifecycle with defer cursor.Close()

Implementing UPDATE Operation​

Now, let's add the UPDATE operation to modify existing users:

app/model.go (Add UPDATE operation)
// UpdateUser updates a user's information
func UpdateUser(id string, update map[string]interface{}) (*User, error) {
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	// Convert string ID to ObjectID
	objectID, err := primitive.ObjectIDFromHex(id)
	if err != nil {
		return nil, fmt.Errorf("invalid ID format: %v", err)
	}

	// Get MongoDB instance from mongokit manager
	db, err := mongokit.GetManager().GetMongoDb(MongoDBInstance)
	if err != nil {
		return nil, fmt.Errorf("failed to get MongoDB instance: %w", err)
	}

	// Add update time to the update document
	update["updated_at"] = time.Now()

	// Create update document
	updateDoc := bson.M{"$set": update}

	// Update the user
	filter := bson.M{"_id": objectID}
	result, err := db.Collection(UsersCollection).UpdateOne(ctx, filter, updateDoc)
	if err != nil {
		return nil, err
	}

	if result.MatchedCount == 0 {
		return nil, errors.New("no user found with the given ID")
	}

	// Get the updated user
	return GetUserByID(id)
}

Understanding the UPDATE Operation​

Our update operation has several important components:

  1. We use MongoDB's $set operator to update only the specified fields
  2. We automatically update the updated_at timestamp
  3. We check the MatchedCount to ensure a document was actually updated
  4. We return the fully updated document by fetching it after the update

Implementing DELETE Operation​

Finally, let's add the DELETE operation to remove users from the database:

app/model.go (Add DELETE operation)
// DeleteUser removes a user from the database
func DeleteUser(id string) error {
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	// Convert string ID to ObjectID
	objectID, err := primitive.ObjectIDFromHex(id)
	if err != nil {
		return fmt.Errorf("invalid ID format: %v", err)
	}

	// Get MongoDB instance from mongokit manager
	db, err := mongokit.GetManager().GetMongoDb(MongoDBInstance)
	if err != nil {
		return fmt.Errorf("failed to get MongoDB instance: %w", err)
	}

	// Delete the user
	filter := bson.M{"_id": objectID}
	result, err := db.Collection(UsersCollection).DeleteOne(ctx, filter)
	if err != nil {
		return err
	}

	if result.DeletedCount == 0 {
		return errors.New("no user found with the given ID")
	}

	return nil
}

Understanding the DELETE Operation​

The delete operation:

  1. Converts the string ID to a MongoDB ObjectID
  2. Uses the DeleteOne method to remove a single document
  3. Checks the DeletedCount to confirm a document was actually deleted
  4. Unlike SQL databases with soft delete, this is a permanent delete operation

Step 4: Create the Controllers​

Now, let's create the controller that will handle HTTP requests for our user API. First, let's create a basic structure for our controller in the controller.go file:

app/controller.go (Basic Structure)
/*
Create By Gonyx Framework

Copyright © 2025
Project: gonyx-mongo-db
File: "app/controller.go"
------------------------------
*/

package app

import (
	"net/http"
	gohttp "github.com/Blocktunium/gonyx/pkg/http"
	"github.com/gin-gonic/gin"
)

// UserController - a controller to handle user-related HTTP requests
type UserController struct{}

// GetName - return the name of the controller to be used as part of the route
func (ctrl *UserController) GetName() string { return "User" }

// Routes - returning controller specific routes to be registered
func (ctrl *UserController) Routes() []gohttp.HttpRoute {
	return []gohttp.HttpRoute{
		{
			Method:    gohttp.MethodGet,
			Path:      "/",
			RouteName: "listUsers",
			F:         ctrl.ListUsers,
		},
		{
			Method:    gohttp.MethodGet,
			Path:      "/:id",
			RouteName: "getUser",
			F:         ctrl.GetUser,
		},
		{
			Method:    gohttp.MethodPost,
			Path:      "/",
			RouteName: "createUser",
			F:         ctrl.CreateUser,
		},
		{
			Method:    gohttp.MethodPut,
			Path:      "/:id",
			RouteName: "updateUser",
			F:         ctrl.UpdateUser,
		},
		{
			Method:    gohttp.MethodDelete,
			Path:      "/:id",
			RouteName: "deleteUser",
			F:         ctrl.DeleteUser,
		},
	}
}

Implementing Controller Methods​

Now let's implement each of the controller methods one by one, starting with the LIST and GET operations:

app/controller.go (LIST and GET operations)
import (
	"net/http"
	"strconv"

	gohttp "github.com/Blocktunium/gonyx/pkg/http"
	"github.com/gin-gonic/gin"
)

// ListUsers - Get all users with pagination
func (ctrl *UserController) ListUsers(c *gin.Context) {
	// Parse pagination parameters
	limit := 10 // Default limit
	skip := 0   // Default skip

	if limitParam := c.Query("limit"); limitParam != "" {
		if parsedLimit, err := strconv.Atoi(limitParam); err == nil && parsedLimit > 0 {
			limit = parsedLimit
		}
	}

	if skipParam := c.Query("skip"); skipParam != "" {
		if parsedSkip, err := strconv.Atoi(skipParam); err == nil && parsedSkip >= 0 {
			skip = parsedSkip
		}
	}

	// Get users from database
	users, err := ListUsers(limit, skip)
	if err != nil {
		c.JSON(http.StatusInternalServerError, gin.H{
			"error": "Failed to get users: " + err.Error(),
		})
		return
	}

	c.JSON(http.StatusOK, gin.H{
		"users": users,
		"count": len(users),
		"limit": limit,
		"skip":  skip,
	})
}

// GetUser - Get a single user by ID
func (ctrl *UserController) GetUser(c *gin.Context) {
	id := c.Param("id")
	if id == "" {
		c.JSON(http.StatusBadRequest, gin.H{"error": "User ID is required"})
		return
	}

	// Get user from database
	user, err := GetUserByID(id)
	if err != nil {
		c.JSON(http.StatusInternalServerError, gin.H{
			"error": "Failed to get user: " + err.Error(),
		})
		return
	}

	if user == nil {
		c.JSON(http.StatusNotFound, gin.H{"error": "User not found"})
		return
	}

	c.JSON(http.StatusOK, user)
}

Implementing CREATE Operation​

Next, let's implement the CREATE operation in our controller:

app/controller.go (CREATE operation)
// CreateUser - Create a new user
func (ctrl *UserController) CreateUser(c *gin.Context) {
	// Parse user data from request body
	var user User
	if err := c.ShouldBindJSON(&user); err != nil {
		c.JSON(http.StatusBadRequest, gin.H{
			"error": "Invalid user data: " + err.Error(),
		})
		return
	}

	// Basic validation
	if user.Name == "" || user.Email == "" {
		c.JSON(http.StatusBadRequest, gin.H{
			"error": "Name and email are required",
		})
		return
	}

	// Check if user with the same email already exists
	existingUser, err := GetUserByEmail(user.Email)
	if err != nil {
		c.JSON(http.StatusInternalServerError, gin.H{
			"error": "Failed to check for existing user: " + err.Error(),
		})
		return
	}

	if existingUser != nil {
		c.JSON(http.StatusConflict, gin.H{
			"error": "User with this email already exists",
		})
		return
	}

	// Create new user
	createdUser, err := CreateUser(user)
	if err != nil {
		c.JSON(http.StatusInternalServerError, gin.H{
			"error": "Failed to create user: " + err.Error(),
		})
		return
	}

	c.JSON(http.StatusCreated, createdUser)
}

Implementing UPDATE and DELETE Operations​

Finally, let's implement the UPDATE and DELETE operations:

app/controller.go (UPDATE and DELETE operations)
// UpdateUser - Update a user's information
func (ctrl *UserController) UpdateUser(c *gin.Context) {
	id := c.Param("id")
	if id == "" {
		c.JSON(http.StatusBadRequest, gin.H{"error": "User ID is required"})
		return
	}

	// Parse update data
	var updateData map[string]interface{}
	if err := c.ShouldBindJSON(&updateData); err != nil {
		c.JSON(http.StatusBadRequest, gin.H{
			"error": "Invalid update data: " + err.Error(),
		})
		return
	}

	// Remove ID field if present in update data
	delete(updateData, "id")
	delete(updateData, "_id")
	// Remove timestamps as they are managed by our code
	delete(updateData, "created_at")
	delete(updateData, "updated_at")

	// Update user
	updatedUser, err := UpdateUser(id, updateData)
	if err != nil {
		c.JSON(http.StatusInternalServerError, gin.H{
			"error": "Failed to update user: " + err.Error(),
		})
		return
	}

	c.JSON(http.StatusOK, updatedUser)
}

// DeleteUser - Delete a user
func (ctrl *UserController) DeleteUser(c *gin.Context) {
	id := c.Param("id")
	if id == "" {
		c.JSON(http.StatusBadRequest, gin.H{"error": "User ID is required"})
		return
	}

	// Delete user
	err := DeleteUser(id)
	if err != nil {
		c.JSON(http.StatusInternalServerError, gin.H{
			"error": "Failed to delete user: " + err.Error(),
		})
		return
	}

	c.JSON(http.StatusOK, gin.H{"message": "User deleted successfully"})
}

Step 5: Register the Controller in App​

Now that we have implemented our controller, we need to register it in the application. Update the app.go file to register our UserController:

app/app.go
/*
Create By Gonyx Framework

Copyright © 2025
Project: gonyx-mongo-db
File: "app/app.go"
------------------------------
*/

package app

import (
	"github.com/Blocktunium/gonyx/pkg/engine"
)

// App - application engine structure that must satisfy one of the engine interface
type App struct {}

// Init - initialize the app
func (app *App) Init() {
	// Register the UserController
	engine.RegisterRestfulController(&UserController{})
}

This simple initialization code does the following:

  1. It registers our UserController with the Gonyx framework
  2. This makes all the routes we defined in our controller available at the base path /api/User/
  3. No additional MongoDB initialization is needed, as Gonyx will handle connecting to MongoDB based on our configuration

Step 6: Testing the Application​

Now that we have set up our MongoDB integration, let's run the application and test our API endpoints.

Starting MongoDB​

Make sure your MongoDB server is running. If you don't have MongoDB installed, you can start a MongoDB instance using Docker:

docker run --name mongodb -p 27017:27017 -d mongo:latest

Starting the Application​

Run your Gonyx application with the development configuration:

go run main.go dev

Your API should now be accessible at http://localhost:8080/api/User/ (assuming the default HTTP port is used).

Testing the API Endpoints​

Let's test each of our CRUD operations with curl commands to ensure everything is working as expected.

Create a New User​

curl -X POST http://localhost:8080/api/User/ \
  -H "Content-Type: application/json" \
  -d '{"name":"Jane Doe","email":"jane@example.com","age":28}'

This should return a response with the newly created user, including the MongoDB ID and timestamps.

List All Users​

curl -X GET "http://localhost:8080/api/User/?limit=10&skip=0"

This should return a list of all users in the database with pagination information.

Get a User by ID​

# Replace this with an actual ID from a created user
curl -X GET http://localhost:8080/api/User/60a2b5e3f1d1c71234567890

This should return the specific user details if found.

Update a User​

# Replace this with an actual ID from a created user
curl -X PUT http://localhost:8080/api/User/60a2b5e3f1d1c71234567890 \
  -H "Content-Type: application/json" \
  -d '{"name":"Jane Smith","age":29}'

This should return the updated user information.

Delete a User​

# Replace this with an actual ID from a created user
curl -X DELETE http://localhost:8080/api/User/60a2b5e3f1d1c71234567890

This should return a success message if the user was successfully deleted.

Understanding MongoDB Integration in Gonyx​

Key Differences from SQL Databases​

When working with MongoDB in Gonyx, you should understand these key differences compared to SQL databases:

  1. Document-Oriented Model: MongoDB stores data as flexible JSON-like BSON documents, which allows for complex nested structures
  2. Schema-less Architecture: Unlike SQL databases, MongoDB collections don't enforce a strict schema
  3. ObjectID Primary Keys: MongoDB automatically generates ObjectID values for primary keys (the _id field)
  4. BSON Query Language: We use BSON documents for queries rather than SQL statements
  5. Atomic Operations: MongoDB provides atomic operations at the document level

Working with MongoDB ObjectIDs​

One important aspect of MongoDB is understanding how to work with ObjectIDs:

  1. ObjectIDs are 12-byte unique identifiers that include a timestamp component
  2. They're represented as 24-character hexadecimal strings in APIs
  3. In our code, we convert between string representation and the native primitive.ObjectID type
  4. When creating new documents, MongoDB generates the ID automatically

Extending the Application​

Now that you have a basic MongoDB integration with Gonyx, here are some ways to extend your application:

Adding Indexes for Better Performance​

// Add this to your application initialization
func InitializeIndexes() error {
    db, err := mongokit.GetManager().GetMongoDb(MongoDBInstance)
    if err != nil {
        return err
    }
    
    // Create an index on the email field for faster lookups
    indexModel := mongo.IndexModel{
        Keys:    bson.D{{Key: "email", Value: 1}},
        Options: options.Index().SetUnique(true),
    }
    
    _, err = db.Collection(UsersCollection).Indexes().CreateOne(context.Background(), indexModel)
    return err
}

Implementing Data Validation​

While MongoDB is schema-less, you can implement validation in your application:

func ValidateUser(user User) error {
    if user.Name == "" {
        return errors.New("name is required")
    }
    if user.Email == "" {
        return errors.New("email is required")
    }
    // Add email format validation, etc.
    return nil
}

Implementing Relationships Between Collections​

In MongoDB, you can implement relationships in two ways:

  1. Embedded Documents: For one-to-few relationships
type Address struct {
    Street  string `bson:"street" json:"street"`
    City    string `bson:"city" json:"city"`
    Country string `bson:"country" json:"country"`
}

type User struct {
    // Existing fields...
    Addresses []Address `bson:"addresses" json:"addresses"`
}
  1. Document References: For one-to-many or many-to-many relationships
type Post struct {
    ID        primitive.ObjectID `bson:"_id,omitempty" json:"id"`
    UserID    primitive.ObjectID `bson:"user_id" json:"user_id"`
    Title     string             `bson:"title" json:"title"`
    Content   string             `bson:"content" json:"content"`
    CreatedAt time.Time          `bson:"created_at" json:"created_at"`
}

Conclusion​

In this tutorial, we've learned how to integrate MongoDB with a Gonyx application by:

  1. Configuring MongoDB connection settings in our application
  2. Creating document models with appropriate BSON tags
  3. Implementing CRUD operations using the MongoDB Go driver
  4. Creating a RESTful API controller for our MongoDB documents
  5. Testing our API endpoints

MongoDB's flexibility makes it an excellent choice for applications that need to store complex, unstructured, or evolving data schemas. The Gonyx framework provides a clean abstraction for working with MongoDB through its mongokit package, allowing you to focus on your application logic rather than database connectivity concerns.

Additional Resources​