Episode Transcript
[00:00:00] Speaker A: Today's episode is sponsored by Elacraft AI. What if you could compress weeks of due diligence work into minutes? Helacraft AI makes this possible with the first closed AI powered platform designed for allocators that automatically generates comprehensive due diligence reports with institutional grade accuracy. Get investment grade diligence without the manual grind.
Check them out at Allcraft AI. That's a L, L O C R A F.
And be sure to tell them Angelo sent you.
[00:00:36] Speaker B: We know we generally where we want to go, we know how much risk we want to take, we know the tracking area, how much we deviate from the benchmark, right? Like why don't I have my, you know, you know, have, have my army of agents, right? And then our decision making is essentially just, you know, around again, risk tolerance, understanding our employers, understanding the stakeholders, understanding things that they don't want to invest in and then tell the agent to.
[00:00:59] Speaker C: Go to work, right?
[00:00:59] Speaker B: And then, you know, kind of report back. I think that's the other branch where it's sort of okay, helping us insource things that we can't insource for potentially a lower cost and then you know, concentrating on the areas where we still always going to need to have fund selection.
[00:01:16] Speaker A: Welcome to the Institutional Edge, a weekly podcast in partnership with Pensions Investments. I'm your host, Angelo Calvello. In each 30 minute episode I interview asset owners, the investment professionals deploying capital, who share insights on carefully curated topics. Occasionally we feature brilliant minds from outside of our industry driving the conversation forward. No fluff, no vendor pitches, no disguise marketing. Our goal is to challenge conventional thinking, elevate the conversation and help you make smarter investment decisions. But always with a little edginess along the way.
So this is the final episode in our AI and institutional investing series. And our topic today is Leapfrogging to Multi Agenic Systems the future of pension fund management. A topic that I might have inspired in a paper I wrote called Investment Management in a Box. In this paper I argued that it is conceptually possible to use multiple agents to replicate an investment management business. And my guest today, Antonio Rodriguez, well he picked up on this idea and he made it his own. Antonio is the Director of investments at building services 32 BJ benefit funds, a $10 billion multi employer pension and benefits fund serving commercial and residential building workers throughout New York, Massachusetts, Connecticut and the east Coast. Antonio's bio is in the show notes. Antonio, thank you for joining me today.
[00:02:43] Speaker B: Absolutely, thank you for having me, Angelo. Really excited to discuss this topic.
[00:02:47] Speaker A: Thank you. And if you permit me. You know, we're talking about kind of a technical topic today. I'd like to provide a little background and maybe set the table because some of our audience members just may not be familiar with AI agents. And I'm going to start by saying you've got these LLMs from Anthropic and ChatGPT, and these are basically chatbots where an AI agent, I mean, it's built on an LLM, but it's a program that could actually go out in the world and do something. So let's say you're going to use ChatGPT to help you with a plan, a trip, an itinerary. Well, it can give you personalized suggestions for hotels and sites and maybe even a daily itinerary, whereas an agent would be able to do all that and it would book your flights and your hotels for you. And like an LLM, an agent is initialized with human prompts that contain detailed descriptions of their behavior, their functions, their memories and their relationships. And what's relevant to us today is that these agents can be combined in several different ways to work together to perform a variety of tasks. And these multiegenic systems, they act much like humans. They collaborate, they communicate, they negotiate, and they adapt in real time to changes in the environment. They perform tasks, they solve problems, and they achieve complex goals that would challenge any single agent. And we're seeing more and more companies beginning to incorporate these agents into their business models and business plans. A PwC survey of 300 executives in May found that 88 say their teams are planning to increase their AI related budgets in the next 12 months due to agenic AI. One clear example that I always see is Toyota. They built a multi agenic system called qbeya, not big on the name, and they use it to speed up the design of powertrains. And it's working. So we've got this powerful transformative technology, yet we don't see broad adoption in our own industry.
So Antonio, to you. How are you thinking about agents and their adoption in the institutional investment industry and specifically maybe even in your own business models?
[00:04:59] Speaker B: Angelo, I do really appreciate kind of the introduction and kind of before I dive into it, I wanted to really discuss this from the point of view of an allocator. And many times you think about allocators, particularly in the pension world, whether multi employer, maybe to a lesser extent, the public world, where we've always been kind of tech light, I guess is the best way to think about it, that oftentimes you think about technology adoption. Kind of writ large. Whether it was really going back to the SaaS platforms, to portfolio analytic tools, things like that. We're oftentimes a generation or so behind. And you could say that the procurement process or our ability to essentially take risk when it comes to technology, that level of adoption, we've always had issues as organizations because I've never necessarily that pressure either it's the procurement, procurement policies or taking technology risk not being necessarily part of our mandate. But I see agents as a huge opportunity.
[00:05:59] Speaker C: Right.
[00:06:00] Speaker B: Because what are the typical complaints for many of the investment offices either getting in the public realm or ones that are in the multi employer space is that we are people like that. Essentially we have a tough time hiring oftentimes because of salary and compensation considerations. It's the first one and then I would say second is that because of that we can't hire and we have a harder time hiring. We're always kind of behind the times, we always are kind of overworked. I think being tech light in this case actually is kind of an opportunity, right. I liken it to kind of the mobile revolution where ideas and let's say I was reading things about places in like Kenya, things like that, where you could kind of leapfrog the landline movement and go straight to mobile. And I think about that for our organizations that ultimately given that we were behind many of the, I guess the more I guess kind of tech forward allocators that we're able to actually not have some of the problems and some of the legacy issues again, whether it's kind of SaaS, whether it's having to have this massive Bloomberg terminal and paying lots of money to integrate all that information.
To me, I think that's where the opportunity lies really. The question is can our governance, can our decision making, can our processes, will we allow them to get out of the way essentially get out of the way for technological innovation?
And what are those particular barriers again in our processes and our governance? And how can we be thoughtful about moving forward? How do we get our trustees, how do we get other staff members comfortable with in kind of the early going, outsourcing, some of the processes and even a little bit of the decision making long term to agents with the right guardrails in place?
[00:07:47] Speaker A: From my perspective, you're thinking about you're starting from sort of a deficiency and you're able to leapfrog into the next generation of AI in my view, these multiegenic systems. And if I understood you're right, you're being driven by how would I Say resource constraints in many ways. And you're looking to overcome those resource constraints with artificial intelligence.
[00:08:11] Speaker B: Absolutely. And I think that oftentimes resource constraints are the engine for innovation, or they can be. I should say they're not always. Right. They can be the engine for innovation. And I oftentimes think about that for organizations in our situation where if you cannot necessarily pay for particular talent and you don't have necessarily the line items to hire just a ton of bodies.
[00:08:35] Speaker C: Right.
[00:08:36] Speaker B: That what are the ways that you can overcome that? And I think for us, and why I'm interested in kind of exploring is that first, what are the areas that are difficult to hire and train for? Or there's impatience around that. And then from there again, how do you build a particular guardrails in place from a governance standpoint?
Interesting about all of that is our portfolios are no less complex. And I think that's where things get. You can solve for it the other way.
[00:09:04] Speaker C: Right.
[00:09:05] Speaker B: You can solve for our resource deficiencies by having a less complex portfolio. In some places have.
Since you were not going to have the number of line items, we're not going to move in alternatives, we're not going to do co investments, we're not going to do. I call it endowment envy. We're not going to move into the endowment model. A few places have done that.
But our portfolios are no less complex as a whole.
[00:09:25] Speaker C: Right.
[00:09:26] Speaker B: And so you have to go one of two ways. Right. And then if you want to have that complex portfolio, I think increasingly with. With all of the data, if you think about how much of even a person in the strategy realm's job is operations, if you think about somebody who, whether in its analyst, how much it's gathering information, how much it's logging into portals, how much is answering the same questions over and over again, looking up particular performance, standardizing performance. There are so tasks for anybody, even in the strategy realm, certainly in places like ours where everybody has to be kind of a generalist, but even at larger organizations like a public pension fund that may have hundreds of staff, I don't know if you'd be surprised at how much of their job is still very much really kind of operational data gathering. And it makes it very difficult to do the analytical work that you are ostensibly paying people to do.
[00:10:20] Speaker A: So you're talking about operational efficiency being achieved. You're overcoming the resource constraints that you might have.
You're dealing with a technology that could help with the complexity of portfolios. And I don't know how important this is to you, but you could also do 24 hour monitoring, you know what I mean? These things, they don't sleep when your staff, your colleagues, they go home. So it does kind of make a lot of sense just speaking about it from a pension fund perspective as being quite additive.
[00:10:53] Speaker B: Absolutely.
[00:10:54] Speaker A: So if you had this system, how would you use it? I mean, we know it's some of the, you know, you're overcoming some barriers, you've got some benefits, but how do you think you'd use it?
[00:11:04] Speaker B: So I think, and this is something that I think is kind of most important in the beginning, right? It's, it's, I think really around some of the, what you would call kind of the ancillary tasks.
[00:11:15] Speaker C: Right.
[00:11:15] Speaker B: And those can be kind of twofold. The first, I would call your ops, not necessarily investment applications, but applications that support the investment function, I think.
[00:11:28] Speaker C: Right.
[00:11:28] Speaker B: And so those fall into two categories, right. Where there's all of the legal work that you're doing, like the first pass, right. Like for instance, we get a ton of, you know, you get all your LPAs, your sub docs, you get all of that information that either a, you're, you're, you know, hiring, you know, someone to fill out, you have an analyst that's doing it, you may outsource some of that, you may have in some cases a law firm doing that where paying law firm fees in order for that to happen. I think that that first level is being able to fill in the, the repetitive tasks, you know, whether it's kyc, where it's aml and so taking it like post decision, right. So post decision around the investment or fund selection process.
[00:12:08] Speaker C: Right.
[00:12:09] Speaker B: And talking about this from a fund allocator, obviously look different allocators that manage some money in house. But as a fund allocator, the first would be okay. After the decisions made, what are all the tasks that are channeled?
[00:12:19] Speaker C: Right.
[00:12:20] Speaker B: And so you have a few, you have, you have to the account opening with a, with the custodian as one example, if you have a portfolio analytics system, you have to put in, you know, hey, make sure that fund is created there as well, like analogous to the custodian. So you can build that feed. How is it categorized? What part of the portfolio is it in what strategy and how do you category that, what the benchmark is. So putting all of that stuff into place, right? You have again, you could trigger, you know, kind of kyc, AML subdocs, LPA side letter negotiations. You're essentially triggering a bunch of tasks, some of which don't need a ton of negotiation and are repetitive. Again, for the most part, you know, the sub docs are going to be 95% the same. There might, you know, a manager could ask a couple of relevant questions that are different than what you saw in Fund three.
[00:13:08] Speaker C: Right.
[00:13:08] Speaker B: But for the most part those are, those are incredibly repetitive but they add on to the kind of the task burden for a small office.
[00:13:17] Speaker C: Right.
[00:13:19] Speaker B: In our office we have about five people who are working on all of this.
[00:13:22] Speaker C: Right.
[00:13:22] Speaker B: And again, with some help from admins and things like that. Some people pinch hitting, but it's a core group of five and having meetings where okay, we get to the portfolio analysis after we go because the money has to get sent. All of this stuff precedes the actual analysis on your decision making. So that's one area where immediately I think the amount of time that it would save is massive. Especially as those. If we can get the costs down tremendously. Performance, right. Like performance reporting, the generation of reports on a monthly and quarterly basis. Nothing out of the box looks really good.
[00:14:02] Speaker C: Right.
[00:14:02] Speaker B: I can't tell you and I imagine this is something that this may how you feel as well with regard to design.
But the amount of time spent on templates and design and changes that trustees want to see or on a quarterly basis, how do you visualize information and the thinking behind it? Having somebody do that thought process for you and there's part of that already done. Like I've used Copilot before to help design reports.
[00:14:28] Speaker C: Right.
[00:14:28] Speaker B: Or design presentations. There's still a lot of work to be done. I think an agent can take that to the next level where it's not just the design, but now you're doing design and content and you're not going to have to do kind of a really thorough pass on whether or not the content is correct. Yes, there's hallucinations, but being able to get 75 or 80% there, I think that's long term, that's being conservative.
[00:14:54] Speaker C: Right.
[00:14:54] Speaker B: So I think those two right away today, performance and the task after fund selection would increase the productivity of our team an untold amount so that we spend more time talking about the portfolio and less time talking about all of the tasks that precede conversations on whether or not we made the right decision.
[00:15:16] Speaker A: It's interesting the way you lay this out. It aligns with a lot of the research I've seen in this space where they talk about staging. You don't want to build what I talked about, the whole box. You want to stage this and you want to Deal with some of the more. I think you used the term operational kind of you've made the funding decision, now we have to fund and go through this process. Is that how you're thinking about this? Conceptually you would stage this and work your way into it?
[00:15:45] Speaker B: Yeah. And I think there's a lot of practical reasons why you would want to stage this.
[00:15:49] Speaker C: Right.
[00:15:49] Speaker B: And I'm going to actually kind of work backwards on this a little bit.
[00:15:52] Speaker C: Right.
[00:15:52] Speaker B: Because you know, if you're thinking about like what is the actual biggest reason I'm going to use my fund in an ERISA context before a public context, because I think it crystallizes the issue more. The ultimate fiduciary is the trustees. At the end of the day and legally there has to be a human, certainly not necessarily in certain process loops, but at the end of the process a human has to be there and that human is. Or those people are multiple steps removed from any of these processes. Right. Like they hire, they have staff, those staff have consultants, we hire managers, managers pick securities. So they're five, six, seven steps removed from this process.
[00:16:33] Speaker C: Right.
[00:16:34] Speaker B: And so it's already not necessarily opaque, but it's not transparent, it's translucent.
[00:16:40] Speaker C: Right.
[00:16:41] Speaker B: It's a translucent process for the trustees.
Adding in a agent or kind of adding AI writ large to the loop may take something that's translucent to make it opaque.
That's the ultimate barrier.
[00:16:55] Speaker C: Right.
[00:16:55] Speaker B: And so you have to stage it.
[00:16:57] Speaker C: Right.
[00:16:57] Speaker B: And you can get away with that if the returns are good. You can get away with almost anything if the returns are good. Honestly, that's why I think tech light, like US stock market does well, you can get rid of like we have managers are still using, you know, Excel based trade tickets.
[00:17:09] Speaker C: Right.
[00:17:09] Speaker B: Like you can get away with all of that stuff if, with being technology light if the returns are there. Right. And I honestly think that's the other barrier towards adoption.
So because returns are paramount in our industry, the best way to introduce technology, I believe is through areas that people don't necessarily think of as having an immediate effect on the risk and return in a portfolio. I'd argue that it does, right? But I think that there is because it's a few steps removed from the hey, how much did we make versus our peers versus the benchmark and what was the volatility of the portfolio? I think that that is why starting in areas where they may not have the immediate impact, where it's sort of, oh, this is the lack of better word. This is the grunt work, right? This is the grunt Work that has to get done in order for us to proceed. But it is really seen as a barrier to the processes that actually get us to what pays our beneficiaries, which is the returns on the assets they contribute on a monthly basis.
[00:18:11] Speaker A: Okay, so I get the staging. I understand your focus. But blue sky with me here, man, where would you go? Because let's assume that there's been some metrics, those metrics have been met on the operational side, Legal's good with this. You brought up an important point. You put up the guardrails. There's a human in the loop. The human's gonna. If you decide to have ultimate decision making and responsibility, somebody's gotta own it. Let's assume all that's working. Okay, where do you go from here? I mean, would you take it to the investment decision making process? I mean, what would you do?
[00:18:43] Speaker B: I'd take it up to that brink.
[00:18:46] Speaker C: Right.
[00:18:46] Speaker B: Because if I'm thinking about, right, let's get to the committee room. Right, let's bring it to the committee room. You know, you have the four or five folks in the table, whoever's on the ic, plus your observers. And maybe it's your board, maybe it's discretionary at the staff level. I'd take it almost up to that, that point.
[00:19:05] Speaker C: Right.
[00:19:06] Speaker B: From a, a selection standpoint, again, I'm going to selection, but I'll talk about kind of rebalancing and tackle allocations later.
And everyone's working off the same particular information. It's not you. You try and remove some of the biases. I don't know if it would. It would ultimately make the committee decision, but it would narrow the pipeline. It would certainly narrow the pipeline. I'd be go so far to say that the top of the funnel all the way to the options presented at committee, I think can ultimately be done by those agents where you kind of give it the guidelines and you go, I think from a rebalancing and tactical asset allocation standpoint, the humans things have become the guardrails.
[00:19:48] Speaker C: Right.
[00:19:48] Speaker B: And so you're setting the limits on how much risk you want to take. And so let's say you have a target of, broadly speaking, 50% global equities, you know, you know, 25% fixed income, 25% alts, just as a, that you set the bands and then the agent is able to trade within those particular bands.
[00:20:10] Speaker C: Right.
[00:20:10] Speaker B: That, that ultimately say, here's how much capital we're going to need for to pay benefits. And the agent decides where you, where you source it from. If you're not, if you don't have enough cash to pay that money, I do think long term you get there where it's again you are. Because the humans at the end of the day have to own the risk. The humans have to be the risk managers.
[00:20:30] Speaker C: Right.
[00:20:30] Speaker B: Again, at least for our purposes. I won't speak for other allocators, but certainly for allocator whereby there is a strong fiduciary obligation that risk management because again the legal risk from a participant's perspective, the cash at the end of the day, if we have a shortfall, the benefits are paid for by either cash from the employers or lower benefits over time for our members. That if human owns a risk, the humans have to be the risk managers. But everything else in between I think you can ultimately give to an agent. So research, for instance, certainly like I got 14, you know, that's, that's, that's like 14 tabs on this window open. I got probably 45 tabs of different research that I'm going to get to, that I'm going to get to.
[00:21:14] Speaker C: Right.
[00:21:14] Speaker B: Like everything from, you know, our private credit managers, their kind of latest with regard to, you know, kind of where they think interest, like all that stuff we have, we get so much research and I'm never able to collate it. So my research function. Absolutely. And then you like the agent prepares my daily briefing and. Or prepares my day, the daily briefing for all of my analysts.
[00:21:36] Speaker C: Right.
[00:21:36] Speaker B: Because reporting. Absolutely. Operations, trade, all that like I believe all of that stuff long term, if we're blue skying it, that task design, task implementation and then ultimately even the agents can necessarily have suggestions.
[00:21:51] Speaker C: Right.
[00:21:52] Speaker B: Like if they're kind of fully autonomous. And that's where I think we do have to be careful. It's kind of reporting.
The one question I have on that.
[00:22:02] Speaker C: Right.
[00:22:02] Speaker B: And I'd love to get your thoughts on this too, is how do we train the next level of decision makers if they're not doing those kind of repetitive tasks to learn. Right. Like I think I've learned more about our portfolio when I was first starting out by data entry. I learned about the portfolio through data entry. The repetition of. All right, here are the names of the managers, here's how they're going to interact. Like how do you train people when the training. You're now saying, okay, we're going to train the agent that shows up all the time already.
I do worry a little bit about what effects that has on junior staff if they're not in some cases learning alongside the agents. And finding a way to have them almost be like agent apprentices in some ways. How could you do something like that? Maybe ask the agent to make a training model for new decision makers. But, but I do think in the beginning, getting off again, all the kind of communications operations, tactical asset allocation to an agent to spend more time on the strategic, having a strategic focus, stakeholder engagement, I think are going to be the areas where more and more those skills are going to be for being able to kind of communicate all of that to your stakeholders, whether it's board or beneficiaries or the folks contributing to the plan.
[00:23:12] Speaker A: I'm going to give you three comments if I could remember them all at my age. First is there's actually a VC firm, I believe based in Hong Kong that now has an AI agent on its investment committee.
[00:23:25] Speaker B: Oh, wow.
[00:23:26] Speaker A: Second, you talked about tactical asset allocation and you're moving away from the operational side into the investment decision making side. I read a couple papers last week and one of them in particular stood out for me. They built a hedging agent and the research looked really good to me. I'm not going to endorse it, but it was peer reviewed. This paper was peer reviewed before it was accepted it into a rather prestigious symposium. But man, I didn't think we had come that far that fast. But I'm already seeing these agents acting in a way that I thought we were a ways away from. I'll admit my naivete. So going to your taa, I mean that's basically kind of a risk decision within your bands. I mean you pick the periodicity, your liquidity needs, et cetera. So this stuff is happening now.
[00:24:17] Speaker B: Oh man.
[00:24:18] Speaker A: The third point is you asked me what about staff? You learned really from the ground up and you were looking at spreadsheets and you were talking to your mentors at the time. I'm not sure we're going to see those same types of individuals hired. I'm more inclined to think you might be hiring someone that's more fluent in data than they are in investing. You may be hiring people that are fluent as prompt engineers and understand retrieval augmented generation.
So I don't want to say you're going to lose these jobs, but the skills that go into those jobs that are required to do those jobs just might be different if you move to a multigenic system.
[00:25:01] Speaker B: Absolutely. And I almost think that it comes into two forms.
[00:25:03] Speaker C: Right.
[00:25:04] Speaker B: Folks who are. It's almost in so many areas in our society. Right. I'll say that the middle.
[00:25:09] Speaker C: Right.
[00:25:10] Speaker B: The generalist right. The that middle kind of goes away as you move to specialist again either in the data realm or in the heart of the communication realm.
[00:25:17] Speaker C: Right.
[00:25:17] Speaker B: Because again, at the end of the day, if there's a. Because, you know, and we are agents, right? Like, you know, I'm an agent, right. Of the principles.
[00:25:24] Speaker C: Right.
[00:25:25] Speaker B: And so being able to communicate to the principal, you know, I'm an agent that needs sleep, I'm an agent that needs to go home, play with his kids, everything like that. But I'm an agent, right. Of my principles. And being able to communicate to those principles, I think is still right now the skill that is necessary. So you have to have, I think you're going to move to one of those kind of barbells kind of one or the other. And so I think that that's going to look there. But the other thing I'm concerned about, and this is something that we always another barrier to technology adoption is who.
[00:25:54] Speaker C: Right.
[00:25:55] Speaker B: Like if you're, if the idea is that we're all convinced that at some point, let's say we're all convinced that pension management a box that the, that having multiple agents, multiple specialized agents to run kind of your, you know, at least, at the very least operations and even some kind of investment in strategic decision making in, at your pension fund makes sense.
[00:26:13] Speaker C: Right.
[00:26:14] Speaker B: But there, you know, unless we believe there's going to be one kind of vendor for all of this. The other concern I have is especially in a relative game like what we, that we play, the vendor selection becomes paramount.
[00:26:26] Speaker C: Right?
[00:26:26] Speaker A: Yeah.
[00:26:27] Speaker B: And already.
[00:26:27] Speaker C: Right.
[00:26:28] Speaker B: Like again, I understand enough to be able to do a portfolio analytics system, right. Like, even if I don't necessarily know everything on the back end, I know what I want, right. With regard, I know kind of what I want to see visually. I know what's easy to work with, what's not easy to work with.
[00:26:41] Speaker C: Right.
[00:26:41] Speaker B: So I can, given, given the reps, I can understand what I'm looking for and then how to decide on which vendor to choose for this. Right. I also know what I want but I think the level of understanding is one that I'm, you know, we have to kind of, we have to get to and I'm not necessarily confident that again, you're talking about particular skill sets. That certainly is folks in our allocators need to have. Right. Like and because it's relative, right. And especially if you move into things like again, trading in fund selection right now, are your agents going head to head? Are you going to head to head with somebody else's Agent certainly again one on on kind of the trading end from an excess return standpoint right now it's sort of if it's that battle that arms rates for agents do folks like us get left in the dust, right. Even if we're forward and adopting how much money are we going to throw at the versus again other allocators or other investors who from their perspective, that's how they make their money.
I work for, I've always worked for places that the money's there, money's contributed from employees and employers. So I don't have the profit motive and I don't necessarily have the motive the kind of risk of fundraising in order to push me to say we have to get the best agent possible, we want to do a good job for our beneficiaries. But I think about that a lot with regard to vendor selection is really does the vendor selection move from who has the best due diligence process to who is able to either work with or prompt an agent to do better. Right. And that becomes I thinking about this new relative game because again from a excess return standpoint or a peer benchmarking standpoint. Right. Like we are in a relative game. Even though we should sometimes act like we are more an absolute game, we're in a relative game when it comes to investment management.
[00:28:35] Speaker A: A quick break to talk about Ellacraft AI. I've been following how AI can transform due diligence and find most platforms give allocators more data, not better decisions. Helacraft AI flips the screen script by providing decision ready intelligence that actually moves the needle instead of drowning in dashboards. You get the tedious work done for you, summarizing documents, flagging risks and writing IC memos so your team can focus on what matters most, making those critical high conviction calls. Transform your pipeline from backlog to competitive of advantage at Alacraft AI. Go to Alacraft AI to schedule a demo and discover the AI due diligence platform allocators deserve.
You're getting into the issue of implementation here and you've kind of conceded you're not going to build this internally. You already admitted early on your resource constraints. So you want to find a supplier that is going to be able to meet your, you know, your initial and your future needs. You want to make sure the suppliers got enough Runway that they don't go out of business in six months. But you also want to be sure in this relative game that they're not using your data to train their own models on. So other people could use that. There's a data privacy issue.
Let's talk about some of the other issues, though. Not implementation, but just using agents. You already mentioned hallucinations.
[00:30:05] Speaker C: Right.
[00:30:06] Speaker A: And we've got this issue of data privacy. That's a big one. Yeah, I could keep going, but hey man, you're the guest, not me.
[00:30:13] Speaker B: Well, here's what interesting, I think, and one of the benefits for us, right. And you see this, and I'm going to use an example on how much easier it is on the asset side. Because I think about the pension world kind of writ large. I'm on the asset side, I'm on the investing side, but I'm on the senior leadership team. We have conversations on the benefit side as well. How much easier it is on our side than on the benefit side.
[00:30:34] Speaker C: Right.
[00:30:35] Speaker B: The costs on the benefit side from a technological implementation is an order of magnitude larger than it is on the investment side. Benefit management systems.
We're in a world that's somewhat standardized, Right. The regulatory guidelines, whether federal or state, I think make it so that there's a lot of standardized information. It's how you use that information.
I worry less about data privacy for us because if you want to look up everything that we invest in, you could do it. You go to the Department of Labor, right. Like we have to list everything. When I was at the public, we had to listen in our annual financial reports, right. It's all there.
So you can look in the section process. When I worked in, you know, Nevada, you could go to the meeting, right. Like it's, you know, see the agenda. Right. So from a allocator's perspective, I'm less worried about data privacy than I would be on the benefit side. But the truth is that like, at the end of the day, we are moving lots of money to beneficiaries sometimes who are, you know, they're of retirement age or older, you know, we're all on the same network.
[00:31:35] Speaker C: Right.
[00:31:36] Speaker B: And so, you know, it's kind of less issue if it's cloud based and stuff like that. But I guess my bigger take is that we may be a gateway to information that, you know, people who want to do harm or steal money want to get to.
[00:31:47] Speaker C: Right.
[00:31:48] Speaker B: Like again, ours will be like account numbers, wiring instructions, things like that, right. Where those games can kind of play. And that's where in some ways we haven't talked about the asset managers kind of on their side. How much humans are going to be in the loop there when they're, when there's that risk of, you know, either accepting money from customers you don't know, or sending distributing proceeds back to particular someone that you don't know. I guess my take is. So, yeah, I'm less worried about kind of the data privacy from the investment standpoint. I'm worried about the data privacy from us being a gateway to folks getting in and taking a beneficiary's check.
[00:32:27] Speaker C: Right.
[00:32:27] Speaker B: That would be where we have to think about the cybersecurity holistically as a big issue.
[00:32:33] Speaker A: Okay, so you've hit one I'm going to throw out there. There's these technical issues. You've got model collapse possibilities with agents. And there was a paper that Anthropic put out maybe two or three weeks ago, and it talked about something called agenic misalignment, where models from, regardless of the developer, they resorted to malicious insider behavior where they were basically blackmailing officials and leaking sensitive information.
And there's a brilliant data scientist, Gary Marcus, he looked at this from a technical perspective and said, nobody in the industry knows how to stop this. I mean, it's one of the problems with this. So it's basically a security vulnerability that we're looking at here. And I would assume when you're going through your procurement process, you're asking your prospective vendors to say, how do you deal with hallucinations, model claps, agent misalignment? And then you're in a position where you have to make that decision or you hire a consultant that sits next to you at the meeting. That may be more, I'll say, technically competent in this area. And no disrespect, but you're not a data scientist.
[00:33:44] Speaker B: No, I'm not a data scientist. And in fact, this is more about.
First, we'd hire a consultant for it. Right.
We're not making this decision often enough to hire someone to have somebody on staff who's looking at this.
[00:33:57] Speaker C: Right.
[00:33:57] Speaker B: For me, even our IT folks. And maybe long term that happens, maybe this gets integrated long enough where your IT staff, certainly they know more, but the idea that they're going to be able to do a search for particular agents, again, we're just using, from the LLM perspective, chatbots for things like client service or member services to answer questions about your summary plan, document or point you in the right direction. That I think is an area where we're already seeing those changes. And, and again, I think, because so much of our, from their perspective, their bread and butter is in our members. But I think that for us, it would be very much, you have to hire a consultant to do it. And then kind of what your trust is. And then the real question is for us, is there enough expertise and cost effective expertise to move that way? And then the justification at the end of the day, how much better returns can we get? Because I think we've talked through all and why it's so much, much easier to talk about kind of the burden, the operational burden versus the investment decision making is because there are real costs, right? Whether in time or in, you know, actual kind of person hours at additional consultants lag opportunity costs for having a long implementation process for onboarding a fund, a longer search process, taking longer to rebalance.
All of that decision making has quantities is easily quantifiable. But from an investment and selection standpoint, right. One thing I think about is we don't do a great job, I think internally of measuring the counterfactual. I'd love to have a counterfactual agent that essentially you're running and making decisions because there's a tree effect, right? Like let's say I said I wanted X small cap manager or S private equity manager, not Y. But essentially having agents kind of run another portfolio. But there are decisions on top of those decisions, right? Like if I make X decision, that tree, that binomial tree ends up moving further and further away from each other over time. I'd love to kind of play with that. We don't do a great job of measuring the counterfactual in our kind of current processes. We don't really have time. But I say I said that why at the end of the day it is much more difficult to kind of measure like we diversified and honestly, if we would just had like, you know, if we had all our money in Nvidia, right. You know, benefits would have improved even more.
[00:36:26] Speaker C: Right?
[00:36:27] Speaker B: Like, but you know, we have all that volatility, right. Sometimes the simplest answer is the best one. And so I guess my point on that one is it is harder to kind of measure it or it's much more multifaceted than in the operations where I think the clear case for automation through agents is already there, right? Like, and the idea of like whether or not you want to be the first mover. But I could, if I snap my fingers tomorrow, can have it all. I could go to the board and say, here's how much money I think we'll save on odd. Here's how much money I think we could save from having con, like you know, multiple layers of consultants. Here's how much time I think we can save from fun decision to onboarding Right. Like that, that is something that we can do. And, and I think that's why the. It's easier to make that case. But you know, we know there's a lot of luck in investment management.
And so, you know, you can do all the right things. Right. Of a process that's pristine and lose out to somebody who threw a dart at a board.
[00:37:27] Speaker C: Right.
[00:37:27] Speaker B: Like. And so, you know, that I think is why.
[00:37:30] Speaker A: Yeah, but it's the process that matters in your seat.
I know it. I mean, I'm vice chair of a little police pension board here and I see. Oversee the investments. I keep telling the officers it's all about process.
If we were that good, you guys wouldn't be police officers. We'd all be running our own heavy or trading offices.
[00:37:50] Speaker B: 100%.
[00:37:51] Speaker A: But it goes back to your comment about staging. Again, you're thinking of this. Yeah, it's a leapfrog, but it's a leapfrog where you're not saying it's all done at once. We're going to stage this. And it's interesting, in your last comments, you talked about how the board from governance structures would say, yeah, we can see where this makes sense with the right guardrails, a human in the loop. And you could say we can actually save money and improve staff's use of time because governance is always an issue. But by staging it and demonstrating empirically that there's actual value here, it's easier to take it to the next stage.
[00:38:27] Speaker C: Absolutely.
[00:38:28] Speaker A: There's one other point about the counterfactual. I mean, there are multigenic systems that build in critics and those critics could play the role. We invested in manager A, but how do we do if we would have done it in manager B?
And you know, so, you know, you're, you're thinking about it conceptually, but it's already being done. Man, you're seeing this already.
[00:38:48] Speaker B: One other thing, and I don't know if you've seen any research on this, and I'm thinking about this and I was going back to sometimes in like our big managers, our large managers, especially the ones attached to kind of the custody banks thing like that, they are, you know, you know, we complain about them sometimes. They're tech, more technology forward.
[00:39:04] Speaker C: Right.
[00:39:04] Speaker B: I can trade on their platforms, everything like that. Some of our boutique managers, they're less technology forward than we are.
[00:39:10] Speaker C: Right.
[00:39:10] Speaker B: Like in the sense of. And so I also think about, we have other organizations that are part of this kind of chain and are they going to be willing to interact in some Cases with agents. So again, giving you the fund opening example where let's say I took and something's being done now. But like let's say I wanted to automate the process from, you know, I got a capital call, just the simple capital call, right. And I'm able to, you know, but, and let's say I, I, I messed up the fund docs, right. And so the wire instructions change. Are they going to be willing to interact with an agent?
[00:39:44] Speaker C: Right, yeah.
[00:39:45] Speaker B: And so then even if we have in internal processes that can be automated and kind of putting in the particular instructions, are they going to want to do a callback?
[00:39:56] Speaker C: Right.
[00:39:56] Speaker B: And if they're, if they, and if they want to do a callback.
[00:39:58] Speaker C: Right.
[00:39:59] Speaker B: For, for a particular large instruction.
[00:40:01] Speaker C: Right.
[00:40:02] Speaker B: While we've automated some of it, you know, the human is coming in the, and more.
[00:40:06] Speaker C: Right.
[00:40:07] Speaker B: And so there's this interactive effect where that can be another limiting factor. Like you can be a fir one to be a first mover. And again, because so much of what we do is not about gaining an advantage against another market player, but about implementation of our own decisions in that aspect. There's sometimes not a ton of benefit of being too far forward of everyone else because there's not a ton of gains because maybe some of our other vendors aren't as technology for it and they're going to force us to put a human back in the loop.
[00:40:40] Speaker C: Right.
[00:40:40] Speaker B: And so we have to kind of stage that a little bit.
[00:40:42] Speaker A: So it's two things. One, you point out another implementation barrier. I mean APIs, they often lack the granular functionality. And second, you may include in your manager diligence, you know, are they AI savvy? I mean you would have your own part of your operational due diligence. Can they do this or are we going to get faxes still? Do we have to communicate by faxes? Are there going to be callbacks? So you would include that in there. And that may not be a reason not to do it, not to invest with the manager, but at least you'll know in spite of these, let's say limitations in terms of your multigenic system, you still want the manager. We're just going to live with that, that, that human interaction.
[00:41:25] Speaker B: Yeah, absolutely. And, and, but this is, this is another thing I think, and this is, I, I probably should have brought this up before because we, we talked about blue sky, right? I, I, I was, I was, my sky wasn't completely blue. There's a few cl, few clouds in it.
[00:41:38] Speaker C: Right.
[00:41:38] Speaker B: Because I was actually on the blue side, what is the biggest barrier for us, if we think about it, is that the fact that we have to do the funds manager selection process in the first place, right. Like if I look at my portfolio analytics system, you know, if I look at, at, at my overall portfolio across, you know, public equities, fixed income alts, right. Like, like I'm the market plus or minus what, 25 basis points, 50 basis points, right. Like you're going to be around that benchmark over a longer period of time, honestly, if the agents can get us closer to being able to essentially just run our fund internally, right. Like that's, and I didn't think about it from it again, was still kind of in the mindset as an allocator, if we think about these particular systems, you know, and you'd still do the fund selection in areas that are most important. But, you know, know, if I look up and I say, okay, I'm basically running a enhanced index, right? Which is again, big picture. That's what we're running for most of allocators of our size, of our regulatory framework. If we're running an enhanced index in both fixed income and in equities, you know, and I'm paying a lot of money to run that in hand indexed, right? So why not? You know, and the barriers are that, you know, well, you know, I'm not a Wall street guy, right, Like I grew up in the allocator realm, right.
But we know generally where we want to go, we know how much risk we want to take, we know the tracking area, how much we deviate from the benchmark, right? Like why don't I have my, you know, have my army of agents, right? And then our decision making is essentially just, you know, around again, risk tolerance, understanding our employers, understanding the stakeholders, understanding things that they don't want to invest in and then tell the agent to.
[00:43:11] Speaker C: Go to work, right?
[00:43:12] Speaker B: And then, you know, kind of report back. I think that's the other branch where it's sort of okay, helping us insource things that we can't insource for potentially a lower cost and then concentrating on the areas where we're still always going to need to have fund selection.
[00:43:28] Speaker A: So going back to the staging though, Antonio, why not continue to do what you're doing in that investment decision making framework with humans, but let the agents kind of work on their own, but let them paper trade, let's say, because the baseline they have to beat is you and your team. You could give the agents the same constraints that the team has to operate under tracking error, sharp liquidity, it doesn't matter. I mean, you build these constraints in and you're prompt and you run it side by side. And after six months, a year, two years, you say, you know what, the agent stinks or the agent's doing a good job, we're going to let it nibble now a little bit more. So you run it side by side is what I would think. Think.
[00:44:09] Speaker B: No, absolutely. And then the other side of it is almost, you know, I don't know if you've seen some of the, the Chessbots, right? They'll make a Magnus Carlson chess bottle. They make like a. Right, you can almost make a trusty bot. Right. I'd love to make a, a trustee bot where it's sort of, oh, you have to go in front of X trustee and Y trustee this month.
[00:44:25] Speaker C: Right.
[00:44:25] Speaker B: Like, right. Might be getting too far of a field here, but, but ultimately, you know, you're kind of, you kind of playing it all through. Like if you're going to work under our constraints, you really got to work under our constraints.
[00:44:35] Speaker C: Right.
[00:44:36] Speaker B: And, and, and so you'd have, okay, here's XH agent, right. And to me that, I think that makes a ton of, a ton of sense. But I think really what we need is to not be afraid to be a, if not a first mover than a very quick second mover. Right? I do think that.
And again, I think the way you get there is staging. The way you get there is kind of the immediate cost savings and time savings in operations. And again, to your point about staging the all on, you know, you can ask it to kind of build a paper like, but hey, they're making no decisions, right? Like you're making no decisions. But at the same time, even if they're just paper trading or even if they're doing, you can learn from them.
[00:45:16] Speaker C: Right.
[00:45:16] Speaker B: Like, I think that the, the idea that having an agent alongside of you to, to say, have you considered this? Here are the biases that you all have in, in your decision making. We all do, right. You know, from a, you know, here's the size of manager. I'm more into fundamental, more into quantitative.
I like core satellite completely.
[00:45:32] Speaker C: Right.
[00:45:32] Speaker B: Like I think core satellite, you know, kind of beta and a little bit of excess return across every single asset class to me is the, is the best starting point. That might not be right. That might not be right. My rebalancing ranges might be too narrow and you know, we should kind of widen them out. Or alternatively, I like to take chips off the table too fast. Right. Like you know that that is those are all kind of built in through an experience. And I think the last step is. And I think this is something I've talked about and it's not just technology, but I think technology in general agents specifically help solve for this. I think the investment management industry has done a pretty good job of solving for the general that essentially the solutions developed get you 80 to 90% there. But how really to think about how we should construct a portfolio and the information we should be taking in.
[00:46:23] Speaker C: In right.
[00:46:24] Speaker B: Who are our members? Our members are like you said at the beginning, the commercial and residential real estate workers in mostly in New York, but up and down the East Coast.
[00:46:31] Speaker C: Right.
[00:46:31] Speaker B: And we have security officers including the, the, the one that passed away unfortunately at the, the.
[00:46:36] Speaker C: The.
[00:46:37] Speaker B: The shooting at.
On Park Avenue was one of our members.
[00:46:40] Speaker C: Right.
[00:46:41] Speaker B: And we need to solve for them.
[00:46:44] Speaker C: Right, right at the end of the day.
[00:46:45] Speaker B: And then the employers who contribute. Right, the employers contribute. We need to solve for them.
And how do we solve for them?
Where okay, where are the risks in our portfolio? That again on the margins or here are the things that they particularly construct me a portfolio where and I think some of the building trades do this on an ad hoc basis where it's sort of like a dollar contributed by helping us our members get jobs actually is incredibly helpful. You can't make it the entire portfolio but there is a number that makes it work. Or when I did New York City, I always had this idea that you know what is the margin in which a an investment in a building in New York both has the return potential but also the tax base.
[00:47:26] Speaker C: Right.
[00:47:26] Speaker B: Or kind of the like that that ultimately gets you there. And again that's a thought exercise. But those are thought exercises that I'm maybe not skilled enough of a researcher or have the time to do but you know agent that has the entire Internet and the smartest minds resources at their disposal. And so to your idea about there's the prompt and how to get it but then there's more time being able to think about the question.
[00:47:50] Speaker C: Right?
[00:47:51] Speaker B: More time and you thinking about like of all this reading. The more time to think about the question the appropriate question to ask and I'd say that the most exciting thing about that is that gives myself my team more time to think about the question and the appropriate question is the thing that then we have the entities, the agents, technology to help us get the answers.
[00:48:12] Speaker C: Answer.
[00:48:12] Speaker A: Let's end it here. I've taken up more time than I I thought I would, but with you, man, this is a great conversation.
[00:48:18] Speaker B: No, no, I appreciate it.
[00:48:19] Speaker A: It was great. I mean, there's got to be a paper here for us to write somewhere, don't you think?
[00:48:23] Speaker B: Oh, man, absolutely. Listen, once we get these agents going, right, like, we'll be able to. To. But, you know, to. To get a. Spend more time asking the questions and writing the papers.
[00:48:31] Speaker A: But I gotta ask you my final question. I asked this to all my guests. Got nothing to do with AI agents, but what's the worst pitch you ever heard of? You've heard hundreds of pitches in your career, man. What's the worst pitch? And don't throw anybody under the bus. I know you wouldn't do it.
[00:48:46] Speaker B: To be fair, most of the time that I get pitches, they're typically institutional. I heard some bad ones. So mine was not necessarily the investment itself. It's how it was done. So this particular firm came through a trustee recommendation, right? And so we had. It was myself and trustee and one other person kind of on the line for an initial pitch.
So we're waiting, and it's probably, you know, you know, five and 10 minutes, and the person has not gotten on yet. And the firms have gotten on yet. Maybe right before we're like, hey, we're leave. Person gets on and he's on the train.
He's the one who set this meeting up. Or they're the one who set the meeting up, right? So they're on the train on their phone, right? So like, not even new laptops. So they're. They're. This is again, an initial intro meeting. Trustee set it up again. Director, investment. Another trustee on the line on, on. On Zoom.
And he goes, oh, you know, I'm so, so sorry that I'm late. I was like, do you want to reschedule? He's like, no, no, no, we're good. We're going to have the conversation. And he proceeds not to discuss the investment opportunity, but asks us for sourcing, right? For leads, right?
In between.
In between.
The go through the tunnel, right? And so cuts out. And so another, like five, ten minutes pass. And so we're all just kind of. We're in different. We're all just texting each other. And then finally gets on from under the tunnel. Is like, I'm so sorry. You know, we had us. It can maybe. I think it's right that we reschedule. And I think we go, you know, I think we're okay. I think we're okay.
You know, you made about the worst first impression that you could possibly make. You're the one who set up the meaning and instead of of discussing the fund opportunity, you wanted to talk to us about like leads, you know, so that was the I think the that was if I went through kind of my it might have been ones that that the actual investment opportunity may have been worse, but that was probably the worst kind of first impression and last impression that that any investment manager that was trying to win our business had.
[00:50:57] Speaker A: So let's hope the lesson was learned by that individual, and I'm sure they did. But Antonio Rodriguez, thank you. Thank you. This was wonderful. This was a pleasure conversation man.
[00:51:07] Speaker B: Absolutely. Thank you and I appreciate the opportunity.
[00:51:11] Speaker A: Thanks for listening. Be sure to visit PNI's website for outstanding content and to hear previous episodes of the show. You can also find us on PNI's YouTube channel. Links are in the show Notes. If you have any questions or comments on the episode, or have suggestions for future topics and guests, we'd love to hear from you. My contact information is also in the show notes, and if you haven't already done so, we'd really appreciate an honest review on itunes. These reviews help us make sure we're delivering the content you need to be successful. To hear more insightful interviews with allocators, be sure to subscribe to the show on the podcast app of your choice. Finally, a special thanks to the Northrup family for providing us with music from the Super Trio. We'll see you next time. Namaste.
[00:52:04] Speaker D: The information presented in this podcast is for educational and informational purposes only. The host, guest and their affiliated organizations are not providing investment, legal, tax, or financial advice. All opinions expressed by the host and guests are solely their own and should not be construed as investment recommendations or advice. Investment strategies discussed may not be suitable for all investors as individual circumstances vary.
[00:52:21] Speaker A: Sam.
