Querying Data from Multiple Data Sources

Remix Conf was full of great experiences, and we are thrilled to share our insights as sponsors and attendees at Remix Conf 2023. This event allowed us to showcase our product, Composabase, and engage with the vibrant developer community. Throughout the conference, we conducted demos, gathered valuable feedback, and explored new ways to enhance our product’s competitive advantage, UI, and usability.

As part of this retro, we dive into the highlights of our journey, focusing on a key theme that resonated strongly with our audience: the ability to take advantage of the unified data layer and execute queries that interact with data from multiple sources.

To illustrate the explanation of "How to query data across multiple data sources," the demo showcased at the RemixConf will serve as the reference, and you can access the Remix application code at Github , while also having the opportunity to try out the deployed application using Cloudflare pages.

What does the demo look like in the Composabase dashboard?

This project it's configured to merge three data sources: PostgreSQL, CockroachDB, and MySQL, which are external databases that we previously had to set up with the proper credentials.

Composabase provides a built-in playground zone where you can use the GraphiQL IDE to query or mutate your data, which is what this looks like.

Sure, the embedded GraphiQL it’s cool, but how does this look on a remix application?

What does the Remix code look like?

Now that we have our unified GraphQL API, we are taking advantage of the awesome GenQL project to generate a SDK client we can use inside the Remix loader to fetch data using the “music” and “movies” namespaces filtering by year and limiting the query results when the user provides those inputs.

In this example, a query taking advantage of the namespace definition for each data source “music” and “movies” provided in the configuration is used to pull data. As you can see, each namespace contains its queries, “findManyAlbum” in “music” and “findManyMovie” in “movies”. All that discovery and generation was made automagically by the Composabase platform when deploying your unified data layer.

export const loader = async ({ request, context }: LoaderArgs) => {
  const url = new URL(request.url);
  const take = url.searchParams.has("take") ? parseInt(url.searchParams.get("take") as string) : undefined;
  const year = url.searchParams.has("year") ? parseInt(url.searchParams.get("year") as string) : undefined;
  const client = getClient(context);
  const {
    music: { findManyAlbum },
    movies: { findManyMovie },
  } = await client.query({
    music: {
      findManyAlbum: {
        __args: {
          take: take,
          where: {
            year: {
              equals: year,
            }
          },
          orderBy: [
            {
              year: "asc",
            },
            {
              name: "asc",
            }
          ],
        },
        ...AlbumFragment,
      },
    },
    movies: {
      findManyMovie: {
        __args: {
          take: take,
          where: {
            year: {
              equals: year,
            }
          },
          orderBy: [
            {
              year: "asc",
            },
            {
              title: "asc",
            }
          ],
        },
        ...MovieFragment,
      },
    },
  });
  return { take, year, movies: findManyMovie, albums: findManyAlbum };
};

The GraphQL client

Since GenQL was used, we can import the “createClient” function from the generated code and then create and expose a new function named “getClient” This function uses several environment values to make the fetch call to our GraphQL API.

import { createClient } from "~/@types/gen";
export const getClient = (context: any) => {
  const { GRAPHQL_ENDPOINT, GRAPHQL_AUTH } = context;
  return createClient({
    url: GRAPHQL_ENDPOINT,
    headers: {
      Authorization: `${GRAPHQL_AUTH}`,
    },
  });
};

The GraphQL fragments

We use fragments to provide reusable pieces of code and avoid duplication in our queries and mutations; the new “MovieFragment” and “AlbumFragment” constants are both types previously generated from the CLI “MovieGenqlSelection” and “AlbumGenqlSelection.” That gives us type-safety, and IDE autocomplete, making the DX way more enjoyable.

import type { MovieGenqlSelection } from "~/@types/gen";
export const MovieFragment: MovieGenqlSelection = {
  id: true,
  title: true,
  director: true,
  year: true,
  synopsis: true,
  genre: {
    name: true,
  },
}
import type { AlbumGenqlSelection } from "~/@types/gen";
export const AlbumFragment: AlbumGenqlSelection = {
  id: true,
  label: true,
  name: true,
  artist: {
    name: true,
  },
  members: true,
  year: true,
  genre: {
    name: true,
  },
}

Rendering the UI

Nothing fancy here, only plain and boring React code; we iterate the data passed from the loader and map the elements to a couple of React components to display the cards shown on the application.

export default function Index() {
  const { take, year, movies, albums } = useLoaderData<typeof loader>();
  return (
    <div className="flex flex-col justify-center w-full py-4">
      <h1 className="text-2xl">All</h1>
      <section className="w-full max-w-7xl">
        <div className="mb-4 flex gap-4 px-4">
        <p className="text-2xl font-bold">Take:</p>
          <TakeCombobox value={take} allowEmpty />
          <p className="ml-auto text-2xl font-bold">Filter by year:</p>
          <YearCombobox value={year} allowEmpty />
        </div>
        <p className="ml-auto text-2xl font-bold">
          {`Music: ${albums.length} of ${take?take:albums.length}`}
        </p>
        <ul className="grid gap-4 grid-cols-1 md:grid-cols-2 lg:grid-cols-3">
          {albums && albums.length > 0 ? (
            albums.map((album) => (
              <li key={album.id}>
                <AlbumCard {...album} />
              </li>
            ))
          ) : (
            <p className="text-2xl font-bold">No albums found</p>
          )}
        </ul>
        <Separator className="my-4" />
        <p className="ml-auto text-2xl font-bold">
          {`Movies: ${movies.length} of ${take?take:movies.length}`}
        </p>
        <ul className="grid gap-4 grid-cols-1 md:grid-cols-2 lg:grid-cols-3">
          {movies && movies.length > 0 ? (
            movies.map((movie) => (
              <li key={movie.id}>
                <MovieCard {...movie} />
              </li>
            ))
          ) : (
            <p className="text-2xl font-bold">No movies found</p>
          )}
        </ul>
      </section>
    </div>
  );
}

We hope this short explanation is useful and answers the most common question during the RemixConf when demoing Composabase “How to query data across multiple data sources.”

Remix Conf 2023 was an incredible platform for us to demonstrate the power and versatility of Composabase, reaffirming the importance of seamless data integration in today’s complex information landscape.

We are immensely grateful for the opportunity to attend Remix Conf 2023, as it provided valuable insights and helped us validate our market fit and identify new areas of opportunity. The developer community’s enthusiastic response and positive reception have brought us one step closer to a successful product release. We are more determined than ever to continue refining Composabase and empowering businesses with efficient data integration solutions.

Stay tuned as we update you on Composabase’s development and future releases. We are excited to embark on this journey with you and create a world where data integration is seamless, intuitive, and impactful.