Javascript

The Evolution of AI Tooling & Ethical AI Practices with Shivay Lamba

Published Oct 1, 2024

Updated Dec 11, 2024

1 min read

Machine Learning and AI expert Shivay Lamba, discusses the evolution of machine learning tools, and his work on MLOps and deploying large language models (LLMs). The conversation covers the accessibility of AI, the power of JavaScript in machine learning through tools like TensorFlow.js, and the growing importance of ethical AI practices. Shivay also discusses the transition of web-based AI tools, the importance of transfer learning, and how developers can break into the space of AI and machine learning.

Chapters

Shivay’s Journey into Machine Learning (00:00 - 03:30)
The Power of TensorFlow.js and Web AI (03:31 - 07:00)
Challenges in Hackathons: Using Pre-trained Models (07:01 - 10:00)
Navigating the AI Ecosystem: Python vs. JavaScript (10:01 - 13:30)
LLMs and Their Growing Popularity (13:31 - 17:00)
The Importance of Core Machine Learning Knowledge (17:01 - 20:00)
AI Ethics & Challenges in Scaling Models (20:01 - 23:00)
Shivay’s Content & Community Involvement (23:01 - 25:00)
Conclusion & Final Thoughts (25:01 - End)

Follow Shivay on Social Media Twitter: https://x.com/HowDevelop Github: https://github.com/shivaylamba

ChatGPT can't solve these problems! with Chris Gardner

Chris Gardner talks neural networks, covering fundamental concepts, historical development, and practical applications. It’s important to understand the difference between artificial intelligence (AI) and machine learning, and the role of neural networks in solving intricate problems with vast datasets. This conversation centers around the intricacies of training neural networks, particularly as they become more complex with multiple layers. Chris and Rob touch on the fascinating yet daunting nature of neural networks, discussing their ability to learn beyond human comprehension. Turning to the practical side of using neural networks, Chris shares the existence of libraries that exist to simplify the process of building a network, enabling users to input data, specify parameters, and entrust the system with the training. Both caution about the biases inherent in the data and the responsibility associated with working on machine learning models. They address challenges related to ethics, highlighting the difficulties in identifying biases and emphasizing the delicate balance between excitement and caution in the evolving field of machine learning. Listen to the podcast here: https://modernweb.podbean.com/e/chris-gardner/...

Jan 16, 2024

1 min

AIMachine LearningChatGPT

Detect Hand Sign Languages with Tensorflow

Interested in learning how to use Tensorflow to detect hand sign languages in your apps? By the end of this read, you will know how to implement Tensorflow in your application with very simple steps. In our example today, we will be using Vue. What is Tensorflow? Tensorflow is an end-to-end platform _(meaning: delivering complex systems or services in functional form after developing it from beginning to end.)_ used for building Machine Learning applications, and it is also open-source. TensorFlow enables you to build dataflow graphs and structures to define how data moves through a graph by taking inputs as a multi-dimensional array called Tensor. You can read more on Tensorflow here. What is a Model? A model is a function with learnable parameters that maps an input to an output. A well-trained model will provide an accurate mapping from the input to the desired output. Tensorflow Models Tensorflow models are pre-trained models, and there are four defined categories of them: - Vision: Analyze features in images and videos. - Body: Detect key points and poses on the face, hands, and body with models from MediPipe - Text: Enable NLP in your web app using the power of BERT and other Transformer encoder architectures. - Audio: Classify audio to detect sounds. If you want to go into more detail, check out Tensorflow Models. All these models are broken down into subs and for our case, we will be making use of the Body Model which has the hand pose detection we need in order to detect the hand signs. Hand Pose Detection This model used a 2D and 3D multi-dimensional array which enables it to predict the keypoints of the hands. Example of a 2D is [[1,2],[3,5],[7,8],[20,44]] and that of a 3D is [[1,2,5],[3,5,8],[7,8,6],[20,44,100]]. This hand pose detection is a model from the MediPipe as we established above, and it provides us with two model types which are lite and full. The accuracy of the prediction increases from lite to full while the inference speed reduces, i.e. the response time will be slower as the accuracy increases. What do we need? There are a few dependencies we need to get things working, and I also will be assuming that you have your project set up as well. You will need to add these dependencies to the project ` Above, in the commands, you will notice we added a fingerpose. Let's talk a little about what we need the figerpose for. Fingerpose Fingerpose is a gesture classifier for hand landmarks detected by Mediapipe hand pose detection. It also allows you to add your own hand gesture, which means that a gesture that signifies the letter Z can signify Hello based on your fingerpose data. We will see an example of how the data looks in a bit. You can check out fingerpose for more details. Get started We are going to use Vue for this illustration. We will start by looking at the HTML first, and then we will cover the JavaScript. Our Template will be a basic HTML that will have a video tag so we can show a video after getting access to our webcam. Template ` The snippet above shows a div and a video tab. The video is used when we gain access to the webcam. We will now be writing the JS required to initialize the webcam. Script ` We imported two methods from vue: onMounted and ref. The onMounted runs when the page is fully mounted while the ref is used to declare a reactive value to reference the video element. If you look at the video tag in the template, you will notice a ref property. You can check out Template ref and onMounted lifecycle hook. In the openCam function, we first try to test if mediaDevices is available on your browser navigation. >The MediaDevices interface provides access to connected media input devices like cameras and microphones, as well as screen sharing. In essence, it lets you obtain access to any hardware source of media data. This MediaDevice has a method getUserMedia which prompts the user for permission to use a media input. You can find all you need to know about getUserMedia here. From the snippet, we can see that getUserMedia returns a promise, and with that, we can get the media stream as a response using then(). We check if the video element has srcObject or not. If it does we assign the media stream to the srcObject and if not, we convert the media stream to a URL and assign it to the src of the video element. With this Snippet and with a few style, you should have your video showing your awesome face! Introducing Tensorflow and Hand Detection Now that we got our webcam working, we will update the Template and the script in order to detect, predict, and display the alphabet based on the hand sign prediction. The updated HTML should now look like this: ` The div with class name alphabet will display the alphabet based on the hand sign prediction. We will be introducing two(2) new functions, createDetectionInstance and handleSignDetection. Firstly, lets begin with the createDetectionInstance which is an integral part of the hand sign detection and then we will introduce handleSignDetection which predicts and displays the hand sign. ` To be able to detect hand poses, we need to create an instance of the handpose detector, and here, we created a function createDetectionInstance which is an asynchronous function. You can check out this Tensorflow blog to see more details. Now that we have created an avenue to detect hand signs, let us start detecting the hand. In that light, we will be adding a handleSignDetection function. ` The handleSignDetection runs after creating the detection instance. We have a setInterval that runs every 2 seconds (PS: _the 2 seconds timing is arbitrary and can be less or more_) to check if there is any hand sign. We also have a conditional statement to ensure the video element exists, and the detection instance was created accordingly. So, the detector calls a method estimateHands, which tries to predict the hand pose by getting keypoints with values that are either in 2D or 3D (Multi-dimensional Array). If you check your console log, you will see an array of data if any hand pose is detected. Now that we can detect hand poses, we will now add fingerpose that will help predict and display the alphabet based on the hand sign. ` Assuming that our detector sensed a hand, it is time to match this value based on the hand signs we created with the fingerpose. The landmark variable is a 3D array pulled from the hand result's keypoint3D key value. There is also a keypoint as well, which is a 2D value, and both will give the same result. Now, using GE.estimate, we can generate a possible gesture that matches the sign, and a score/confidence is assigned to each gesture pending the amount of gesture predicted. So, the gesture with the highest score/confidence is selected since it is estimated to be the closest to the hand sign from the figerpose hand signs we created. We also imported Handsigns and its content looks like this: You can also get the handsigns folder from the 100-ms-vue repository. Looking at the screenshot, there is a GestureDescription instance that takes a string A which will represent what the hand sign will stand for. So, it could be anything you want the handsign to stand for. >onMounted is asynchronous because we need to ensure that our detection instance is created, which is required to detect the hand sign. With the updated code, you should be able to display some letters. Conclusion Don't forget, you can see in detail how this was implemented in one of This Dot Labs' open source projects 100-ms-vue. Please note that what we did is just a basic implementation, and to have a production-ready version, it will need a bigger model, and a more complex detection to be able to identify hand sign language....

Jul 12, 2023

7 mins

Artificial IntelligenceTensorflowMachine Learning

Introduction to Zod for Data Validation

As web developers, we're often working with data from external sources like APIs we don't control or user inputs submitted to our backends. We can't always rely on this data to take the form we expect, and we can encounter unexpected errors when it deviates from expectations. But with the Zod library, we can define what our data ought to look like and parse the incoming data against those defined schemas. This lets us work with that data confidently, or to quickly throw an error when it isn't correct. Why use Zod? TypeScript is great for letting us define the shape of our data in our code. It helps us write more correct code the first time around by warning us if we are doing something we shouldn't. But TypeScript can't do everything for us. For example, we can define a variable as a string or a number, but we can't say "a string that starts with user_id_ and is 20 characters long" or "an integer between 1 and 5". There are limits to how much TypeScript can narrow down our data for us. Also, TypeScript is a tool for us developers. When we compile our code, our types are not available to the vanilla JavaScript. JavaScript can't validate that the data we actually use in our code matches what we thought we'd get when we wrote our TypeScript types unless you're willing to manually write code to perform those checks. This is where we can reach for a tool like Zod. With Zod, we can write data schemas. These schemas, in the simplest scenarios, look very much like TypeScript types. But we can do more with Zod than we can with TypeScript alone. Zod schemas let us create additional rules for data parsing and validation. A 20-character string that starts with user_id_? It's z.string().startsWith('user_id_').length(20). An integer between 1 and 5 inclusive? It's z.number().int().gte(1).lte(5). Zod's primitives give us many extra functions to be more specific about *exactly* what data we expect. Unlike TypeScript, Zod schemas aren't removed on compilation to JavaScript—we still have access to them! If our app receives some data, we can verify that it matches the expected shape by passing it to your Zod schema's parse function. You'll either get back your data in exactly the shape you defined in your schema, or Zod will give you an error to tell you what was wrong. Zod schemas aren't a replacement for TypeScript; rather, they are an excellent complement. Once we've defined our Zod schema, it's simple to derive a TypeScript type from it and to use that type as we normally would. But when we really need to be sure our data conforms to the schema, we can always parse the data with our schema for that extra confidence. Defining Data Schemas Zod schemas are the variables that define our expectations for the shape of our data, validate those expectations, and transform the data if necessary to match our desired shape. It's easy to start with simple schemas, and to add complexity as required. Zod provides different functions that represent data structures and related validation options, which can be combined to create larger schemas. In many cases, you'll probably be building a schema for a data object with properties of some primitive type. For example, here's a schema that would validate a JavaScript object representing an order for a pizza: ` Zod provides a number of primitives for defining schemas that line up with JavaScript primitives: string, number, bigint, boolean, date, symbol, undefined, and null. It also includes primitives void, any, unknown, and never for additional typing information. In addition to basic primitives, Zod can define object, array, and other native data structure schemas, as well as schemas for data structures not natively part of JavaScript like tuple and enum. The documentation contains considerable detail on the available data structures and how to use them. Parsing and Validating Data with Schemas With Zod schemas, you're not only telling your program what data should look like; you're also creating the tools to easily verify that the incoming data matches the schema definitions. This is where Zod really shines, as it greatly simplifies the process of validating data like user inputs or third party API responses. Let's say you're writing a website form to register new users. At a minimum, you'll need to make sure the new user's email address is a valid email address. For a password, we'll ask for something at least 8 characters long and including one letter, one number, and one special character. (Yes, this is not really the best way to write strong passwords; but for the sake of showing off how Zod works, we're going with it.) We'll also ask the user to confirm their password by typing it twice. First, let's create a Zod schema to model these inputs: ` So far, this schema is pretty basic. It's only making sure that whatever the user types as an email is an email, and it's checking that the password is at least 8 characters long. But it is *not* checking if password and confirmPassword match, nor checking for the complexity requirements. Let's enhance our schema to fix that! ` By adding refine with a custom validation function, we have been able to verify that the passwords match. If they don't, parsing will give us an error to let us know that the data was invalid. We can also chain refine functions to add checks for our password complexity rules: ` Here we've chained multiple refine functions. You could alternatively use superRefine, which gives you even more fine grained control. Now that we've built out our schema and added refinements for extra validation, we can parse some user inputs. Let's see two test cases: one that's bound to fail, and one that will succeed. ` There are two main ways we can use our schema to validate our data: parse and safeParse. The main difference is that parse will throw an error if validation fails, while safeParse will return an object with a success property of either true or false, and either a data property with your parsed data or an error property with the details of a ZodError explaining why the parsing failed. In the case of our example data, userInput2 will parse just fine and return the data for you to use. But userInput1 will create a ZodError listing all of the ways it has failed validation. ` ` We can use these error messages to communicate to the user how they need to fix their form inputs if validation fails. Each error in the list describes the validation failure and gives us a human readable message to go with it. You'll notice that the validation errors for checking for a valid email and for checking password length have a lot of details, but we've got three items at the end of the error list that don't really tell us anything useful: just a custom error of Invalid input. The first is from our refine checking if the passwords match, and the second two are from our refine functions checking for password complexity (numbers and special characters). Let's modify our refine functions so that these errors are useful! We'll add our own error parameters to customize the message we get back and the path to the data that failed validation. ` Now, our error messages from failures in refine are informative! You can figure out which form fields aren't validating from the path, and then display the messages next to form fields to let the user know how to remedy the error. ` By giving our refine checks a custom path and message, we can make better use of the returned errors. In this case, we can highlight specific problem form fields for the user and give them the message about what is wrong. Integrating with TypeScript Integrating Zod with TypeScript is very easy. Using z.infer<typeof YourSchema> will allow you to avoid writing extra TypeScript types that merely reflect the intent of your Zod schemas. You can create a type from any Zod schema like so: ` Using a TypeScript type derived from a Zod schema does *not* give you any extra level of data validation at the type level beyond what TypeScript is capable of. If you create a type from z.string.min(3).max(20), the TypeScript type will still just be string. And when compiled to JavaScript, even that will be gone! That's why you still need to use parse/safeParse on incoming data to validate it before proceeding as if it really does match your requirements. A common pattern with inferring types from Zod schemas is to use the same name for both. Because the schema is a variable, there's no name conflict if the type uses the same name. However, I find that this can lead to confusing situations when trying to import one or the other—my personal preference is to name the Zod schema with Schema at the end to make it clear which is which. Conclusion Zod is an excellent tool for easily and confidently asserting that the data you're working with is exactly the sort of data you were expecting. It gives us the ability to assert at runtime that we've got what we wanted, and allows us to then craft strategies to handle what happens if that data is wrong. Combined with the ability to infer TypeScript types from Zod schemas, it lets us write and run more reliable code with greater confidence....

Nov 15, 2024

7 mins

JavaScriptZod

Svelte 5 is Here!

Svelte 5 was finally released after a long time in development. Fortunately, we've been able to test it for some time, and now it has a stable release. Let's dig into its features and why this is such a significant change, even though Svelte 4 code is almost 100% compatible. Svelte syntax everywhere This is one of my favorite additions. Previously, Svelte syntax was limited to a component element. Refactoring code from a component and moving it to a JavaScript file worked differently. Now you can use the .svelte.js or .svelte.ts extension, which allows you to use the same syntax everywhere. It's important to note that it's a way to express that this is not just JS, and what you write may be compiled to something else, just like .svelte files are not just html even though they look very similar. Runes The introduction of runes is one of the most significant changes in Svelte. Many users felt attracted to variables being instantly reactive in previous versions. ` There was, however, a lot of magic underneath and a considerable amount of work from the compiler to make it behave reactively. $state In Svelte 5, a reactive variable has to be explicitly declared using the $state rune, which brings a clear distinction between what is reactive and what isn’t. ` In the previous code, $state is not an actual function being called; it's a hint for the compiler to do something special with this declaration. Rune names always start with a dollar sign ($) and do not need to be imported. However, there’s much more to the changes than just the way we declare reactive variables. Runes bring a lot of new features that make Svelte 5 a great improvement. In Svelte 5, objects or arrays use Proxies to allow for granular reactivity, meaning that individual properties in an object are reactive, even if they are nested objects or arrays. If you modify a property, it will only update that property and will not trigger an update on the whole object. It also supports triggering updates when methods like array.push are called. In previous versions, an assignment was required to trigger an update: ` To have a similar behavior of svelte 4 (no deep reactivity), use the $state.raw() rune. This syntax of .* is common for related features of a rune: $state.raw() will require an assignment to trigger reactivity. ` Because proxies are used, you may need to extract the underlying state instead of the proxy itself. In that case, $state.snapshot() should be used: ` $derived We will use $derived and $derived.by to declare a derived state. The two runes are essentially the same, except one allows us to use a function instead of an expression, allowing for more complex operations to calculate the derived values. ` $effect Effects are useful for running something triggered by a change in state. One of the things you'll note from the $effect rune is that dependencies don't need to be explicit. Those will be picked from reactive values read synchronously in the body of the effect function. ` Something to bear in mind is that these dependencies are picked each time the effect runs. The values read during the last run become the dependencies of the effect. ` Depending on the result of the random method, foo or bar will stop being a dependency of the effect. You should place them outside the condition so they can trigger reruns of the effect. *Variants* of the effect rune are $effect.pre, which runs before a DOM update, and $effect.tracking(), which checks for the context of the effect (true for an effect inside an effect or in the template). $props This rune replaces the export let keywords used in previous versions to define a component's props. To use the $props syntax, we can consider it to return an object with all the properties a component receives. We can use JavaScript syntax to destructure it or use the rest property if we want to retrieve every property not explicitly destructured.. ` $bindable If you want to mutate a prop so that the change flows back to the parent, you can use the $bindable prop to make it work in both directions. A parent component can pass a value to a child component, and the child component is able to modify it, returning the data back to the parent component. ` $inspect The $inspect rune only works in dev mode and will track dependencies deeply. By default, it will call console.log with the provided argument whenever it detects a change. To change the underlying function, use $inspect.with().with): ` $host The host rune gives access to the host element when compiling as a custom element: ` ` Other changes Another important change is that component events are just props in Svelte 5, so you can destructure them using the $props() rune: ` A special children property can be used to project content into a component instead of using slots. Inside the components, use the [@render[(https://svelte.dev/docs/svelte/@render) tag to place them. ` Optional chaining (?.) prevents from attempting to render it if no children are passed in. If you need to render different components in different places (named slots before Svelte 5, you can pass them as any other prop, and use @render with them. Snippets Snippets allow us to declare markup slices and render them conveniently using the render tag (@render). They can take any number of arguments. ` Snippets can also be passed as props to other components. ` ` Conclusion Some exciting changes to the Svelte syntax were introduced while, at the same time, maximum compatibility efforts were made. This was a massive rewrite with numerous improvements in terms of performance, bundle size, and DX. Besides that, a new CLI has been released, making the whole experience of starting a project or adding features delightful. If you haven't tried Svelte before, it's a great time to try it now....

Nov 27, 2024

4 mins

Svelte

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

The Evolution of AI Tooling & Ethical AI Practices with Shivay Lamba

You might also like

ChatGPT can't solve these problems! with Chris Gardner

Detect Hand Sign Languages with Tensorflow

Introduction to Zod for Data Validation

Svelte 5 is Here!

Let's innovate together!

You might also like

ChatGPT can't solve these problems! with Chris Gardner

Detect Hand Sign Languages with Tensorflow

Introduction to Zod for Data Validation

Svelte 5 is Here!